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1.  Introduction 


1.1  Background 

The  most  common  way  to  present  route  instructions  graphically  is  a  geographic  two-dimensional 
(2-D)  paper  map  that  is  annotated  with  route  information,  either  directly  on  the  map  or  on  an 
overlay.  It  provides  an  overview  of  the  surroundings  and  is  a  more  abstract  rendering  of  the 
user’s  location  than  third-person  perspective  and  augmented  reality  (AR)  maps.  When  paper  2-D 
maps  are  used,  many  users  rotate  the  maps  so  that  the  map  orientation  is  consistent  with  their 
movement  because  they  find  the  mental  rotation  task  challenging. 

Egocentric  map  perspectives  such  as  the  2-D  rotating  map  have  been  found  to  be  more  efficient 
in  route-guiding  situations  than  exocentric  maps  such  as  the  north  up  map  (Porathe,  2007; 
Hermann,  Bieber,  and  Duesterhoeft,  2003).  Wickens  and  Prevett  (1995)  found  support  for  their 
model  of  aircraft  displays  that  proposes  that  local  guidance  is  better  supported  by  more 
egocentric  displays  and  the  global  awareness  is  better  supported  by  less  egocentric  displays. 
Many  users  find  it  useful  for  the  map  to  rotate  into  their  current  viewing/walking  direction 
because  they  believe  that  this  preserves  their  cognitive  resources.  They  do  not  need  to  mentally 
or  physically  turn  the  map  to  be  consistent  with  their  walking  direction,  thus  freeing  their  limited 
resources  for  other  uses.  However,  this  type  of  2-D  map  may  require  more  constant  attention  on 
the  part  of  the  user  to  keep  the  proper  orientation  as  they  navigate  the  terrain.  Redden,  Elliott, 
Pettitt,  and  Carstens  (2008)  found  that  sometimes  when  drivers  using  a  rotating  2-D  map  looked 
away  to  perform  other  tasks,  the  map  rotated  because  the  vehicle  was  gradually  turning  and  the 
Soldiers  had  difficulty  reorienting  and  sometimes  even  did  not  realize  that  the  map  had  rotated 
for  several  seconds.  Seager  and  Stanton-Fraser  (2007)  found  this  to  be  true  when  their 
experiment  demonstrated  that  users  find  it  difficult  to  recognize  a  map  that  rotates  automatically 
when  they  are  not  looking  at  the  map. 

Third-person  perspective  or  three-dimensional  (3-D)  maps  give  a  birds-eye  view  or  view  from  a 
high  position.  Haberlin  (2005)  defines  such  a  map  as  a  “computer-generated  perspective  view  of 
a  3-D  geo-data  model  with  cartographic  content.”  The  axonometric  view  from  these  maps  can 
make  elevation  differences  easier  for  the  user  to  understand,  enhance  the  user’s  understanding  of 
the  spatial  relationships  between  features,  and  create  a  less  abstract  view  of  the  world  than  2-D 
maps.  Rakkolainen,  Timmerheid,  and  Vainio  (2000)  found  some  evidence  that  people  recognize 
landmarks  and  find  routes  in  cities  easier  using  a  third-person  perspective  3-D  model  than  using 
a  2-D  map  and  that  search  and  visualization  was  more  intuitive  with  3-D.  Ubiquitous 
applications  such  as  Goggle  Earth  indicate  the  growing  popularity  of  3-D  maps.  However,  they 
do  have  some  problems.  The  scale  diminishes  from  the  front  to  the  back  of  the  map,  making 
distance  difficult  to  judge,  and  high  topographic  features  can  obscure  adjacent  lowland  areas  and 
slopes  facing  away  from  the  users,  depriving  them  of  critical  information. 
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AR  systems  overlay  geo-spatially  registered  information  on  the  user’s  experience  of  the  real 
world.  The  objects  in  the  user’s  surroundings  become  the  background  for  computer-generated 
annotations  that  “augment”  the  visual  field  with  information  necessary  in  the  performance  of  the 
current  task.  Azuma  (1997)  defines  AR  as  a  system  that  combines  real  and  computer- generated 
information  in  real  time  and  aligns  virtual  objects  with  physical  ones  in  the  environment.  AR 
supplements  reality  rather  than  replacing  it.  AR  can  aid  navigation  and  movement  through  the 
environment  by  pointing  out  locations  in  the  user’s  field  of  view  and  by  directional  annotations 
(e.g.,  arrows  and  routes  drawn  on  the  ground).  AR  offers  other  benefits  besides  navigational 
ones.  For  example,  AR  can  provide  annotated  information  in  the  visual  field  concerning 
landmarks,  people,  or  buildings  seen  in  the  environment.  AR  allows  the  user  to  concentrate  on 
computer  supplied  information  and  the  real  world  at  the  same  time.  AR  might  not  be  as  efficient 
for  planning  purposes  as  2-D  maps  because,  like  third-person  perspective  maps,  topographic 
features  can  obscure  adjacent  lowland  areas  and  slopes  facing  away  from  the  users,  depriving 
them  of  critical  information.  AR  devices  also  limit  planning  distances  to  those  that  can  be  seen 
(line  of  sight  and  closer  distances). 

1.2  Objective 

The  goal  of  this  study  was  to  evaluate  the  cognitive  load  generated  by  four  types  of  digital  maps 
that  have  the  potential  for  use  in  the  augmented  reality/immersive  information  portrayal 
technologies  based  on  the  end  user  device  (EUD).  The  four  types  of  maps  included:  a  2-D  north 
up  (2D-NU)  map;  a  2-D  rotating  (2D-R),  course  up  map;  a  third-person  perspective  (3PP)  map; 
and  an  AR  display. 

1.3  Overview  of  Experiment 

This  study  was  an  investigation  of  the  effect  of  the  four  different  types  of  maps  on  map-based 
task  performance  and  cognitive  load.  It  took  place  at  Fort  Benning,  GA.  Thirty-one  Soldiers 
from  the  Officer  Candidate  School  (OCS)  and  one  Staff  Sergeant  instructor  participated  in  the 
study.  After  training  on  the  simulation  and  the  operation  of  the  Soldier  EUD  and  software,  each 
Soldier  completed  four  scenarios;  one  with  each  of  the  four  different  types  of  maps.  The  map 
order  was  counterbalanced  along  with  the  different  scenarios  to  control  for  the  effect  of  learning. 
Performance  was  evaluated  based  on  objective  performance  data,  data  collector  observations, 
and  Soldier  questionnaires. 


2.  Method 


2.1  Participants 

Thirty-two  Soldiers  were  recruited  from  the  OCS  to  participate  in  the  study.  The  Soldiers 
included  those  with  prior  enlisted  service  who  had  a  variety  of  backgrounds  and  experience 
levels  as  well  as  those  just  coming  into  the  Army  from  college. 
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2.2  Apparatus  and  Instruments 

2.2.1  Apparatus 

2.2. 1.1  Soldier  End  User  Device  Software 

The  Soldier  user  interface  software  employed  during  this  user  jury  (v2.0.1.3)  was  based  on  the 
Nett  Warrior  EUD  software  that  was  used  during  the  Network  Integration  Evaluation  (NIE)  12.1. 
It  was  modified  by  Natick  Soldier  Research,  Development  and  Engineering  Center  (NSRDEC) 
to  include  advanced  navigation  features,  such  as  3PP  and  AR.  It  was  run  using  the  Android 
Gingerbread-2.3.4  operating  system  on  a  Samsung  Galaxy  S  II.  All  four  types  of  digital 
mapping  capabilities  were  integrated  with  this  software.  The  display  size  for  the  phone  was 
4.27  in  and  the  resolution  was  480  x  800  pixels  with  218  pixels  per  inch. 

2.2. 1.2  Maps 

The  maps  used  in  this  experiment  were  geographical-registered  satellite  images,  not  the  typical 
five-color,  2-D  topographic  maps  most  associated  with  Soldier  tasks.  The  first  map  used  during 
this  investigation  (figure  1)  was  a  standard  north  up  digital  map.  Use  of  this  2D-NU  map 
required  the  participants  to  mentally  rotate  the  map  or  to  physically  turn  the  map  to  face  the 
direction  of  travel.  The  second  map  was  the  2D-R  map  (figure  2)  that  revolved/rotated  so  the 
direction  of  travel  was  always  “up”  on  the  screen.  The  third  digital  map  (figure  3)  was  the  3PP 
map,  which  placed  the  users’  perspectives  on  the  terrain  at  about  a  30°  angle  above  the  ground. 
The  fourth  map  (figure  4)  was  an  AR  map.  To  use  the  AR  map,  the  users  placed  the  EUD  in  the 
camera  mode  so  they  could  see  the  terrain  on  the  simulation  through  the  display.  Icons, 
waypoints,  and  other  markings  could  be  seen  on  top  of  the  actual  location  in  the  simulation.  For 
example,  they  could  see  an  improvised  explosive  device  (IED)  icon  directly  above  the  IED  in  the 
simulation.  All  four  maps  were  pre-loaded  with  operational  graphics — route,  checkpoints,  phase 
lines,  objective,  and  building  labels.  Additionally,  the  maps  displayed  an  icon  indicating  the 
users’  positions  based  on  simulated  global  positioning  system  (GPS)  feeds.  The  digital  2-D 
maps  in  this  study  were  updated  as  the  users  moved  so  the  users’  icons  were  always  visible.  The 
2D-NU  did  not  rotate  with  the  users.  For  example,  if  the  users  were  traveling  south,  the  users’ 
icons  moved  downward  on  the  map;  if  the  users  turned  right  while  traveling  south,  the  icons 
turned  left.  The  2D-R  rotated  so  that  the  users’  directions  of  travel  were  always  facing  up.  The 
3PP  map  showed  more  of  a  3-D  view  than  the  2-D  maps  and  the  users  had  a  bird’s  eye 
perspective.  In  this  study,  when  the  end  user  AR  device  was  viewed,  it  showed  the  end  user 
camera  view  with  overlay  labels  of  building  numbers,  chemical  lights  (chemlights),  etc.  It  also 
showed  the  operational  graphic  for  the  planned  route  of  travel. 
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Figure  3.  3PP  screen  shot. 


Figure  4.  AR  screen  shot. 
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2.2.2  Instruments 


2.2.2. 1  Simulation 

The  NSRDEC  First  Person  Virtual  Simulation  that  was  used  in  this  experiment  is  a  portable 
simulation  that  uses  digitized  terrain  from  the  McKenna  Urban  Training  and  Experimentation 
facility,  located  at  Fort  Benning,  GA.  For  this  experiment,  it  was  loaded  onto  a  standard  laptop 
computer.  The  computer  communicated  with  the  Soldier’s  EUD  via  WiFi.  The  virtual 
simulation,  with  simulated  GPS  capability,  allows  the  Soldier  to  move  through  the  terrain  via  a 
joystick  attached  to  the  computer.  In  addition  to  the  digitized  terrain  and  buildings,  the 
simulation  includes  vegetation,  opposing  forces,  civilians  on  the  battlefield  and  objects  of 
interest  such  as  IEDs.  Soldier  position  data  was  logged  on  the  EUD  and  plotted  from  the  EUD 
files,  after  scenario  completion,  to  show  the  Soldier’s  path  of  travel.  The  Soldier  EUD  is  able  to 
track  the  users’  locations  in  the  simulation  and  show  it  and  their  orientation  on  the  maps.  Each 
Soldier  in  the  simulation  plays  the  platoon  leader  in  the  scenario  and  operates  as  a  discrete  unit, 
operating  at  the  Soldier’s  pace.  Squad  members  are  not  shown  in  the  simulation.  The  data 
collector  acts  as  the  company  commander  and  provides  periodic  verbal  reports  and  requests  that 
drive  the  Soldiers’  movements  through  the  digitized  terrain.  Note,  because  this  is  a  simulation, 
there  are  actions  associated  with  tasks  encountered  in  the  scenarios  such  as  “identify  a 
contaminant”  that  cannot  be  conducted  in  the  simulated  environment  (i.e.,  don  a  Chemical 
Biological  Radiological  Nuclear  and  Explosives  suite).  The  simulation  concentrates  only  on  the 
map-based  tasks. 

The  participants  are  first  given  an  operations  order  that  tasks  them  with  finding:  (1)  a  person  of 
interest,  (2)  a  suspected  weapons  cache,  and  (3)  a  chemical  agent.  The  participants  are  provided 
a  picture  of  the  person  of  interest.  Their  digital  maps  provide  them  one  of  four  marked  routes 
into  the  McKenna  facility.  The  participants,  acting  as  the  platoon  leader  in  the  scenario,  receive 
orders  from  the  company  commander  to  mark  items  of  interest  on  the  routes  and  in  the  village 
with  a  digital  icon  and  check  out  activities  along  the  route  and  in  the  village.  When  the 
participants  encounter  civilians  on  the  route  outside  of  McKenna,  they  are  required  to  interrogate 
them  (for  the  purpose  of  this  experiment  in  simulation,  interrogate  meant  to  closely  view  their 
facial  features)  to  determine  if  any  of  them  are  the  person  of  interest.  When  the  person  of 
interest  is  found,  the  participants  report  back  to  the  company  commander.  One  iteration  of  the 
scenario  is  considered  to  be  complete  once  all  tasks  are  attempted. 

The  Soldiers  begin  each  of  the  four  scenarios  approximately  500  m  away  from  McKenna  Village 
in  forested  terrain  and  receive  instructions  to  follow  the  route  to  McKenna  (marked  on  the  map) 
and  report  arrival  at  each  prescribed  waypoint  indicated  on  the  map.  Eight  civilians  on  the 
battlefield  are  present  along  the  route  outside  of  McKenna  for  interrogation  examination 
regarding  the  person  of  interest.  As  the  platoon  leader  reaches  each  check  point/waypoint  on  the 
map,  the  platoon  leader  reports  to  the  company  commander  whose  follow-on  orders  drive  the 
accomplishment  of  the  following  ten  tasks. 
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Task  1 


The  company  commander  calls  in  a  possible  IED,  giving  the  approximate  azimuth  and  asks  the 
platoon  leader  for  confirmation  of  the  IED.  The  platoon  leader  is  told  that  the  bursting  radius  of 
the  IED  is  large  so  its  location  must  be  estimated  from  several  meters  away  (just  close  enough  to 
the  IED  to  confirm  its  location  and  that  it  is  an  IED).  The  platoon  leader  then  moves  to  the 
approximate  IED  location,  identifies/confirms  the  IED,  and  places  a  digital  chemlight  on  the 
EUD  map  marking  the  IED  location.  Once  the  IED  is  marked,  the  platoon  leader  moves  to  the 
next  waypoint. 

•  Measure  1:  The  time  required  to  move  to  the  IED,  drop  the  chemlight,  and  move  to 
waypoint  2. 

Time  begins  at  the  end  of  the  word  “confirmation”  and  ends  upon  reaching  waypoint  2  on  the 
route  after  placing  the  chemlight  on  the  estimated  location  of  the  IED.  Time  is  recorded  by  the 
data  collector. 

•  Measure  2:  The  accuracy  of  the  chemlight  location. 

The  system  logs  the  position  of  the  chemlight  “dropped”  by  the  participant  on  the  map. 

Task  2 

On  the  route,  the  platoon  leader  encounters  concertina  wire  and  has  to  deviate  from  the  route  in 
order  to  navigate  around  the  wire.  The  wire  configuration  turns  the  participants  around  several 
times  to  attempt  to  disorient  them. 

•  Measure  3:  The  time  required  to  negotiate  the  concertina  wire  and  move  to  waypoint  3. 

The  data  collector  begins  timing  once  the  platoon  leader  reaches  the  wire  and  ends  timing  once 
the  platoon  leader  arrives  at  the  next  waypoint,  which  is  on  the  far  side  of  the  wire. 

Task  3 

The  company  commander  calls  and  gives  the  approximate  azimuth  for  a  possible  contaminant 
with  a  large  danger  area  radius  and  asks  the  platoon  leader  for  confirmation.  The  platoon  leader 
moves  to  the  contaminant  location,  locates  the  contaminant,  places  a  digital  chemlight  on  the 
location,  and  returns  to  the  route. 

•  Measure  4:  The  time  required  to  move  to  the  contaminant,  drop  a  chemlight  and  move  to 
waypoint  4. 

The  data  collector  begins  timing  after  the  word  “confirmation”  and  ends  timing  once  the  platoon 
leader  reaches  waypoint  4. 

•  Measure  5:  The  accuracy  of  the  chemlight  location. 

The  system  logs  the  position  of  the  chemlight  on  the  map. 
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Task  4 


Once  the  platoon  leader  reaches  the  route  release  point  outside  of  McKenna,  the  platoon  leader 
receives  a  call  from  the  company  commander  telling  the  platoon  leader  to  establish  a  casualty 
collection  point  (CCP)  in  a  specified  location  and  to  mark  it  with  a  chemlight.  The  company 
commander  gives  a  cardinal  direction  and  a  building  number,  i.e.,  southwest  of  building  B3. 

This  requires  the  platoon  leader  to  use  the  graphic  labels  on  the  EUD  map  display. 

•  Measure  6:  The  location  of  the  icon. 

The  system  logs  the  position  of  the  CPP  chemlight  on  the  map. 

Task  5 

The  company  commander  calls  the  platoon  leader  and  states  that  the  team  has  taken  casualties 
that  need  immediate  care  and  asks  the  platoon  leader  to  move  to  that  location  to  provide  help. 

The  location  is  marked  in  the  graphics  with  a  “call  for  medic”  symbol,  simulating  that  a  digital 
call  for  medic  actually  was  transmitted. 

•  Measure  7:  The  time  required  for  the  platoon  leader  to  move  to  the  call  for  medic  symbol. 

Time  begins  after  the  location  is  given  and  ends  when  the  platoon  leader  reaches  the  correct  grid 
coordinates. 

Task  6 

The  company  commander  calls  the  platoon  leader  and  states  that  shots  were  fired  by  insurgents, 
they  were  seen  running  in  a  certain  direction  (i.e.,  to  NW from  building  C2,  etc.),  and  a  call  for 
fire  (CFF)  is  needed. 

•  Measure  8:  The  time  required  for  the  platoon  leader  to  place  a  digital  chemlight  where  the 
CFF  is  needed. 

Time  begins  after  the  words  “call  for  fire  is  needed”  and  ends  when  the  CFF  chemlight  is  placed. 

•  Measure  9:  The  accuracy  of  the  chemlight  location. 

The  system  logs  the  position  of  the  chemlight  on  the  map. 

Task  7 

The  company  commander  calls  the  platoon  leader  saying  that  the  person  of  interest  has  been 
spotted  in  the  marketplace  relative  to  a  numbered  building  that  is  marked  on  the  operations  order 
map  and  that  the  platoon  leader  needs  to  move  to  that  location  and  look  for  the  person  of  interest. 

•  Measure  10:  The  time  required  to  find  the  person  of  interest. 

Time  begins  when  the  words  “move  to  that  location”  are  stated  and  ends  when  the  person  has 
been  identified. 
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Task  8 


The  platoon  leader  receives  a  call  from  the  company  commander  saying  that  a  potential  weapons 
cache  has  been  found  and  asks  the  platoon  leader  to  move  to  that  location  (i.e.,  northeast  comer 
of  building  x )  to  confirm.  The  platoon  leader  moves  to  the  weapons  cache  location  and  places  a 
digital  chemlight  on  the  location  upon  confirming  the  location. 

•  Measure  11:  The  time  required  to  discover  the  weapons  cache. 

Time  begins  when  the  last  word  of  the  request  is  given  and  ends  when  the  platoon  leader  places 
the  chemlight. 

•  Measure  12:  The  accuracy  of  the  chemlight  location. 

The  system  logs  the  position  of  the  chemlight  on  the  map. 

Task  9 

The  company  commander  calls  the  platoon  leader  saying  that  a  potential  chemical  agents  in  two 
55-gal  drums  located  in  a  certain  position  (i.e.,  southwest  corner  of  building  x )  has  been  found 
and  needs  confirmation.  The  platoon  leader  moves  to  that  location,  confirms  the  presence  of  the 
drums,  and  drops  a  chemlight  symbol  on  the  map. 

•  Measure  13:  The  time  required  to  discover  the  suspected  chemical  agents  (55-gal  drum). 
Time  begins  when  the  word  “confirmation”  is  stated  and  ends  when  a  chemlight  is  placed. 

•  Measure  14:  The  accuracy  of  the  chemlight  location. 

The  system  logs  the  position  of  the  chemlight  on  the  map. 

•  Measure  15:  The  number  of  local  nationals  located  out  of  eight  along  the  route. 

2. 2. 2. 2  Scenarios 

Four  different  scenarios  (A-D)  of  equal  difficulty  were  used  during  this  investigation.  The 
terrain  traveled  to  get  to  McKenna  and  the  buildings  used  in  the  scenarios  were  different  but  the 
tasks  and  the  difficulties  of  the  tasks  and  the  distance  traveled  were  the  same. 

2. 2. 2. 3  Demographic  Questionnaire 

The  demographic  questionnaire  addressed  such  areas  as  navigation  experience,  map  reading 
experience,  digital  system  experience,  and  physical  characteristics  that  might  have  had  an  effect 
on  participant’s  ability  to  operate  the  interface. 
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2. 2. 2.4  Cube  Comparison  Test 


The  Cube  Comparison  Test  (Ekstrom,  French,  and  Harman,  1976)  was  used  to  assess 
participants’  spatial  abilities.  The  Cube  Comparison  Test  requires  participants  to  compare,  in 
3  min,  21  pairs  of  six-sided  cubes  and  determine  if  the  rotated  cubes  were  the  same  or  different. 

2. 2. 2. 5  National  Aeronautics  and  Space  Administration  (NASA)-Task  Load  Index  (TLX) 

The  NASA-  TLX  is  an  assessment  tool  that  allows  subjective  workload  assessments  by 
operator(s)  working  with  various  human-machine  systems  (Hart  and  Staveland,  1988).  It  uses  a 
multidimensional  rating  procedure  that  derives  an  overall  workload  score  based  on  a  weighted 
average  of  ratings  on  six  subscales.  These  subscales  include  mental  demands,  physical  demands, 
temporal  demands,  own  performance,  effort,  and  frustration.  It  can  be  used  to  assess  workload 
in  various  human-machine  environments  such  as  aircraft  cockpits;  command,  control,  and 
communication  workstations;  supervisory  and  process  control  environments;  simulations;  and 
laboratory  tests.  The  version  of  the  NASA-TLX  used  during  this  experiment  was  presented  to 
the  Soldiers  on  a  computer.  Definitions  of  each  scale  were  provided  on  laminated  paper  so  the 
participates  could  refer  to  it  as  they  provided  their  estimates  of  the  workload  associated  with 
each  map  type.  The  definitions  are  shown  in  table  1. 

Table  1.  NASA-TLX  definitions  and  endpoints. 


Title 

Endpoints 

Descriptions 

Mental  demand 

Low/High 

How  much  mental  and  perceptual  activity  was  required  (e.g., 
thinking,  deciding,  calculating,  remembering,  looking,  searching, 
etc.)?  Was  the  task  easy  or  demanding,  simple  or  complex,  exacting 
or  forgiving? 

Physical  demand 

Low/High 

How  much  physical  activity  was  required  (e.g.,  pushing,  pulling, 
turning,  controlling  activating,  etc.)?  Was  the  task  easy  or 
demanding,  slow  or  brisk,  slack  or  strenuous,  restful  or  laborious? 

Temporal  demand 

Low/High 

How  much  time  pressure  did  you  feel  due  to  the  rate  or  pace  at 
which  the  task  or  task  elements  occurred?  Was  the  pace  slow  and 
leisurely  or  rapid  and  frantic? 

Performance 

Good/Poor 

How  successful  do  you  think  you  were  in  accomplishing  the  goals 
of  the  task  set  by  the  experimenter?  How  satisfied  were  you  with 
your  performance  in  accomplishing  these  goals? 

Effort 

Low/High 

How  hard  did  you  have  to  work  (mentally  and  physically)  to 
accomplish  your  level  of  performance? 

Frustration  level 

Low/High 

How  insecure,  discouraged,  irritated,  stressed,  and  annoyed  versus 
secure,  gratified,  content,  relaxed,  and  complacent  did  you  feel 
during  the  task? 

2. 2. 2. 6  Questionnaires 

The  post  iteration  questionnaires  were  designed  to  elicit  Soldiers’  opinions  about  their 
performance  and  experiences  with  each  of  the  map  conditions.  The  questionnaires  ask  the 
Soldiers  to  rate  their  performance  with  the  maps  on  a  7-point  semantic  differential  scale  ranging 
from  “extremely  good/easy”  to  “extremely  bad/difficult.”  The  post  experiment  questionnaire 
consisted  of  questions  asking  the  participant  to  make  comparisons  between  the  map  types  and 
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usability  questions.  Usability  questions  cover  such  areas  as  adequacy  of  training,  difficulty,  and 
intuitiveness  of  the  software,  size  of  windows,  size  of  icons,  placement  of  icons,  ease  of  placing 
icons,  and  ease  of  access  to  information. 

2.3  Procedures 

2.3.1  Soldier  Orientation 

Upon  arrival,  the  participants  received  a  roster  number  to  identify  them  throughout  the 
evaluation.  The  Soldiers  were  given  an  orientation  on  the  purpose  of  the  study  and  what  their 
participation  would  involve.  They  were  briefed  on  the  objectives  and  procedures,  as  well  as  on 
the  operator  interface.  They  were  also  told  how  the  results  will  be  used  and  the  benefits  the 
military  can  expect  from  this  investigation.  Any  questions  the  subjects  had  regarding  the  study 
were  answered.  The  Soldiers  completed  an  informed  consent  form,  medical  status  form,  and  a 
demographics  questionnaire. 

2.3.2  Demographics 

The  participants  completed  a  questionnaire  concerning  their  military  training  and  experience, 
including  their  experience  with  maps  and  global  positioning  systems  (appendix  A). 

2.3.3  Cube  Comparison  Test 

The  Cube  Comparison  Test  was  administered  upon  completion  of  the  demographic  form. 

2.3.4  Training 

During  training,  Soldiers  were  given  training  on  the  operation  of  the  simulation,  on  the  software 
interface  and  on  the  EUD.  Then  they  were  issued  an  operations  order  outlining  the  mission. 
Soldiers  were  trained  on  the  operation  of  the  interface  using  all  map  configurations  and  were 
given  time  to  practice  operating  the  interfaces  with  each  map  type. 

2.3.5  Scenario  Trials 

Upon  completion  of  training,  the  participants  completed  four  iterations  of  the  scenarios,  one  with 
each  map  type  and  scenario  according  to  the  Graeco-Latin  square  matrix  in  table  2.  Scenarios 
are  labeled  A,  B,  C,  and  D. 

Table  2.  Order  of  scenarios  and  map  combinations. 


Roster 

Iteration 

No. 

1 

2 

3 

4 

1,5,9,  13,  17,21 

A/2DNU 

B/2DR 

C/3PP 

D/AR 

2,  6,  10,  14,  18,  22 

D/2DR 

C/2DNU 

B/AR 

A/3PP 

3,7,  11,  15,  19,  23 

B/3PP 

A/AR 

D/2DNU 

C/2DR 

4,  8,  12,  16,  20,  24 

C/SAR 

D/3PP 

A/2DR 

B/2DNU 
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A  data  collector  was  present  with  each  participant  to  record  response  times  (RT)  and  call  in 
requests  (acting  as  the  company  commander).  The  system  logged  the  routes  taken  by  the 
participant  and  the  positions  at  which  they  placed  the  chemlights. 

2.3.6  Questionnaires 

Post  iteration  questionnaires  and  the  NASA-TLX  were  administered  to  each  Soldier  at  the  end  of 
each  iteration  corresponding  to  each  map,  and  the  end  of  experiment  questionnaire  was 
administered  once  all  four  iterations  and  the  associated  questionnaires  were  complete. 

2.4  Experimental  Design 

The  design  of  this  experiment  was  a  single  factor  repeated  measures  design. 

2.4.1  Independent  Variable 

The  independent  variable  was  the  type  of  map. 

2.4.2  Dependent  Variables 

The  following  are  the  dependent  variables: 

•  Times  to  complete  tasks 

•  Accuracy  of  chemlight  placement 

•  Number  of  local  nationals  located  and  interrogated 

•  Identification  of  the  person  of  interest 

•  NASA-TLX  scores 

•  Data  collector  observations 

•  Participant  questionnaire  responses 

2.4.3  Predictor  Variable 

The  predictor  variable  is  the  Cube  Comparison  Test. 

2.5  Data  Analysis 

All  objective  data  collected  were  analyzed  using  repeated  measures  analysis  of  variance 
(ANOVA).  Follow-on  pair  wise  comparisons  were  done  using  Holm’s  Bonferroni  procedure  to 
control  for  family-wise  error  rates  (Holm,  1979).  Partial  eta  squared  (?/  p),  an  index  of  effect 
size,  was  computed  for  each  ANOVA.  Iteration  effects  were  controlled  through  the 
counterbalanced  order  of  the  experimental  design.  Soldier  questionnaire  data  were  analyzed 
using  descriptive  statistics  on  the  subjective  ratings. 
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3.  Results 


3.1  Demographics 

The  31  participants  were  OCS  candidates  at  Fort  Benning,  GA  and  one  was  a  Staff  Sergeant 
instructor.  All  had  received  Initial  Basic  Training  at  a  minimum.  The  average  age  of  the 
Soldiers  was  26  years,  and  the  average  time  in  service  was  19  months.  Seven  of  the  participants 
had  previously  deployed  to  a  combat  area  while  in  the  military.  Twelve  wore  prescriptions 
lenses.  Their  self-reported  experience  with  Army  digital  systems  was  below  average  and  with 
civilian  maps  and  GPS  systems  was  slightly  above  average.  Detailed  responses  to  the 
demographics  questionnaire  are  available  in  appendix  A. 

3.2  Training 

Participants  stated  the  training  they  received  was  thorough  and  fully  prepared  them  to  perform 
the  tasks  required  to  conduct  the  search  mission  in  the  virtual  simulation. 

3.3  Cube  Test  Results 

Table  3  displays  the  participants’  Cube  Test  score  means  that  have  been  separated  based  upon 
the  map  condition  they  chose  as  the  best  overall  condition. 

Table  3.  Cube  Test  means  based  upon  the  overall  map  condition  preference. 


Map  condition 

Mean  number 

correct 

SD 

Number  who  chose  this  as  the 
preferred  map  condition 

2D-NU 

8.86 

5.07 

21 

2D-R 

7.78 

6.55 

9 

3PP 

11.00 

0 

1 

AR 

3.00 

0 

1 

3.4  Scenario  Trial  Results 
3.4.1  Timed  Data 

Tables  4  throughl4  show  the  means  and  standard  deviations  for  each  of  the  timed  measures  in 
the  scenario.  Also  shown  are  the  repeated  measures  ANOVAs  results  for  each  timed  measure 
and  the  results  from  follow-on  pair-wise  comparisons  subsequent  to  the  significant  ANOVAs 
conducted  with  the  use  of  Holm’s  sequential  Bonferroni  to  control  for  family- wise  error  rates. 
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Table  4.  Measure  1.  Times  (sec)  required  to  move  to  the  IED,  drop 
the  chemlight  and  move  to  waypoint  2. 


Map  condition 

Mean 

SD 

2D-NU 

78 

26 

2D-R 

73 

21 

3PP 

81 

32 

AR 

89 

30 

F(3,93)  =  1.89, p  =  0.136,  //'„  =  0.287 

Table  5.  Measure  3.  Times  required  negotiating  the  concertina  wire 
and  moving  to  waypoint  3. 


Map  condition 

Mean 

SD 

2D-NU 

71 

26 

2D-R 

79 

37 

3PP 

87 

40 

AR 

109 

66 

F(3,93)  =  4.33,  p  =  0.007,  //-„  =  0.123 

Table  6.  Paired  samples  test  for  Measure  3. 


Pair 

t 

df 

obtained  p 

required  p 

2D-NU  vs.  2D-R 

-1.46 

31 

0.16 

0.025 

2D-NU  vs.  3PP 

-1.68 

31 

0.10 

0.017 

2D-NU  vs.  AR 

-2.79 

31 

0.01 

0.008 

2D-R  vs.  3PP 

<1 

31 

0.43 

0.050 

2D-R  vs.  AR 

-1.97 

31 

0.06 

0.013 

3PP  vs.  AR 

-2.19 

31 

0.04 

0.010 

Table  7.  Measure  4.  Times  required  to  move  to  the  contaminant,  drop 
a  chemlight  and  move  to  waypoint  4. 


Map  condition 

Mean 

SD 

2D-NU 

65 

27 

2D-R 

66 

24 

3PP 

75 

33 

AR 

96 

55 

F(3,93)  =  5.59,  p  =  0.001,  r\\  =  0.153 

Table  8.  Paired  samples  test  for  Measure  4. 


Pair 

t 

df 

obtained  p 

required  p 

2D-NU  vs.  2D-R 

<1 

31 

0.85 

0.050 

2D-NU  vs.  3PP 

-1.71 

31 

0.10 

0.017 

2D-NU  vs.  AR 

-3.53a 

31 

<0.001 

0.008 

2D-R  vs.  3PP 

-1.15 

31 

0.26 

0.025 

2D-R  vs.  AR 

-2.92a 

31 

0.01 

0.010 

3PP  vs.  AR 

-1.83 

31 

0.08 

0.013 

dp< 0  .05,2-tailed. 
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Table  9.  Measure  7.  Times  required  for  the  platoon  leader  to  move 
to  the  call  for  medic  symbol. 


Map  condition 

Mean 

SD 

2D-NU 

37 

14 

2D-R 

41 

20 

3PP 

42 

18 

AR 

56 

23 

F( 3,93)  =  5.81,  p  =  0.001,  =  0.158 

Table  10.  Paired  samples  test  for  Measure  7. 


Pair 

t 

df 

obtained  p 

required  p 

2D-NU  vs.  2D-R 

<1 

31 

0.34 

0.025 

2D-NU  vs.  3PP 

-1.29 

31 

0.21 

0.017 

2D-NU  vs.  AR 

-3.67* 

31 

<0.001 

0.008 

2D-R  vs.  3PP 

<1 

31 

0.87 

0.050 

2D-R  vs.  AR 

-2.66* 

31 

0.01 

0.013 

3PP  vs.  AR 

-2.84* 

31 

0.01 

0.010 

*  p  <  0.05,  2-tailed. 


Table  11.  Measure  8.  Times  required  for  the  platoon  leader  to  place 
a  digital  chemlight  where  the  CFF  is  needed. 


Map  condition 

Mean 

SD 

2D-NU 

48 

23 

2D-R 

57 

40 

3PP 

50 

40 

AR 

68 

43 

743,93)  =  1-88,  p  =  0.138,  r\\  =  0.057 

Table  12.  Measure  10.  Times  required  to  find  the  person  of  interest. 


Map  condition 

Mean 

SD 

2D-NU 

35 

22 

2D-R 

39 

22 

3PP 

36 

25 

AR 

41 

19 

F  <1 

Table  13.  Measure  11.  Times  required  to  discover  the  weapons  cache. 


Map  condition 

Mean 

SD 

2D-NU 

41 

42 

2D-R 

37 

34 

3PP 

42 

46 

AR 

59 

65 

7X3,90)  =  1.14,  p  =  0.335,  q'„  =  0.037 
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Table  14.  Measure  13.  The  time  required  to  discover  the  suspected 
chemical  agents  (55-gal  drums). 


Map  condition 

Mean 

SD 

2D-NU 

23 

15 

2D-R 

24 

21 

3PP 

25 

19 

AR 

32 

27 

F(3,93)  =  1.24,  p  =  0.299,  pzn  =  0.038 

The  participants  took  more  time  using  the  AR  map  on  every  measure  (figure  5).  Three  of  the 
measures  showed  that  they  were  significantly  slower  with  AR  map  than  they  were  with  the  2D- 
NU  map  and  on  those  measures,  the  comparisons  between  the  AR  map  and  the  2D-R  and  3PP 
maps  were  either  significant  or  they  approached  significance.  Although  the  2D-NU  map  was 
quickest  on  all  timed  measures  but  two,  there  were  no  significant  differences  found  on  any  of  the 
timed  measures  between  the  2D-NU,  2D-R,  and  3PP  maps. 


Figure  5.  Total  task  times. 

3.4.2  Accuracy  Measure  Results 

Tables  15-26  display  the  accuracy  measures  taken  when  the  participants  were  required  to  place  a 
chemlight  on  a  specific  location. 
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Table  15.  Measure  2.  Distance  (meters)  of  chemlight  from  the  IED. 


Map  condition 

Mean 

SD 

2D-NU 

18.1 

10.6 

2D-R 

18.5 

10.4 

3PP 

17.6 

10.4 

AR 

33.4 

32.3 

F(3,90)  =  5.37,  p  =  0.002,  //'„  =  0.152 

Table  16.  Paired  samples  test  for  Measure  2. 


Pair 

t 

df 

obtained  p 

required  p 

2D-NU  vs.  2D-R 

<1 

31 

0.84 

0.025 

2D-NU  vs.  3PP 

<1 

31 

0.84 

0.050 

2D-NU  vs.  AR 

-2.54 

31 

0.02 

0.008 

2D-R  vs.  3PP 

<1 

31 

0.68 

0.017 

2D-R  vs.  AR 

-2.44 

31 

0.02 

0.010 

3PP  vs.  AR 

-2.47 

31 

0.02 

0.013 

Table  17.  Measure  5.  Distance  of  chemlight  from  contaminant. 


Map  condition 

Mean 

SD 

2D-NU 

20.2 

12.5 

2D-R 

20.8 

10.5 

3PP 

15.0 

8.4 

AR 

24.2 

18.0 

F( 3,87)  =  2.68,  p  =  0.05,  q  \  =  0.09 

Table  18.  Paired  samples  test  for  Measure  5. 


Pair 

t 

df 

obtained  p 

required  p 

2D-NU  vs.  2D-R 

<1 

31 

0.76 

0.050 

2D-NU  vs.  3PP 

2.16 

31 

0.06 

0.013 

2D-NU  vs.  AR 

<1 

31 

0.36 

0.017 

2D-R  vs.  3PP 

2.49 

31 

0.02 

0.008 

2D-R  vs.  AR 

<1 

31 

0.43 

0.025 

3PP  vs.  AR 

-2.44 

31 

0.02 

0.010 

Table  19.  Measure  6.  Percent  of  participants  who  correctly 
placed  the  chemlight  in  the  vicinity  of  the  CCP. 


Map  condition 

Mean 

SD 

2D-NU 

84 

37 

2D-R 

78 

42 

3PP 

47 

51 

AR 

3 

17 

F(3,93)  =  29.87,  p  <  0.001,  q\  =  0.49 
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Table  20.  Paired  samples  test  for  Measure  6. 


Pair 

t 

df 

obtained  p 

required  p 

2D-NU  vs.  2D-R 

<1 

31 

0.54 

0.050 

2D-NU  vs.  3PP 

3.22a 

31 

<0.001 

0.017 

2D-NU  vs.  AR 

9.76a 

31 

<0.001 

0.008 

2D-R  vs.  3PP 

2.99a 

31 

0.01 

0.025 

2D-R  vs.  AR 

9.64a 

31 

<.0001 

0.010 

3PP  vs.  AR 

4.91a 

31 

<0.001 

0.013 

a  p  <  0.05,  2-tailed. 


Table  21.  Measure  9.  Distance  from  the  chemlight  placement  to  the  CFF. 


Map  condition 

Mean 

SD 

2D-NU 

55.9 

29.1 

2D-R 

56.2 

27.5 

3PP 

48.1 

26.9 

AR 

149.2 

169.9 

773,93)  =  9.46,  p  <  0.001,  //-„  =  0.234 

Table  22.  Paired  samples  test  for  Measure  9. 


Pair 

t 

df 

obtained  p 

required  p 

2D-NU  vs.  2D-R 

<1 

31 

0.97 

0.050 

2D-NU  vs.  3PP 

<1 

31 

0.33 

0.025 

2D-NU  vs.  AR 

— 3. 16a 

31 

0.01 

0.010 

2D-R  vs.  3PP 

1.40 

31 

0.17 

0.017 

2D-R  vs.  AR 

-2.96a 

31 

0.01 

0.013 

3PP  vs.  AR 

-3.34a 

31 

<  0.001 

0.008 

d  p  <  0.05,  2-tailed. 


Table  23.  Measure  12.  Distance  from  the  chemlight  to  the  weapons  cache. 


Map  condition 

Mean 

SD 

2D-NU 

9.1 

4.4 

2D-R 

8.7 

3.7 

3PP 

9.1 

4.8 

AR 

45.6 

38.4 

7X3,93)  =  28.41,  p  =  <  0.001,  n\  =  0.478 

Table  24.  Paired  samples  test  for  Measure  12. 


Pair 

t 

df 

obtained  p 

required  p 

2D-NU  vs.  2D-R 

<1 

31 

0.69 

0.017 

2D-NU  vs.  3PP 

<1 

31 

0.99 

0.050 

2D-NU  vs.  AR 

-5.25a 

31 

<0.001 

0.013 

2D-R  vs.  3PP 

<1 

31 

0.74 

0.025 

2D-R  vs.  AR 

-5.47a 

31 

<0.001 

0.008 

3PP  vs.  AR 

-5.46a 

31 

<0.001 

0.010 

dp  <  0.05,  2-tailed. 
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Table  25.  Measure  14.  Distance  from  the  chemlight  location 
to  the  55-gal  drums. 


Map  condition 

Mean 

SD 

2D-NU 

9.4 

4.6 

2D-R 

9.2 

3.0 

3PP 

9.5 

4.4 

AR 

27.0 

20.0 

F( 3,90)  =  21.99,  p  <  0.001,  >i\  =  0.423 

Table  26.  Paired  samples  test  for  Measure  14. 


Pair 

t 

df 

obtained  p 

required  p 

2D-NU  vs.  2D-R 

<1 

31 

0.82 

0.025 

2D-NU  vs.  3PP 

<1 

31 

0.90 

0.050 

2D-NU  vs.  AR 

-4.6  r 

31 

<0.001 

0.013 

2D-R  vs.  3PP 

<1 

31 

0.73 

0.017 

2D-R  vs.  AR 

-4.89a 

31 

<.001 

0.010 

3PP  vs.  AR 

-4.99a 

31 

<0.001 

0.008 

d  p  <  0.05,  2-tailed. 


Figure  6  shows  the  average  distance  from  the  target  that  the  chemlights  were  placed  in  each  map 
condition  for  all  the  accuracy  measures. 


60 


2D-NU  2D-R  3PP  AR 


Map  condition 


Figure  6.  Mean  distance  from  target,  chemlight  placement. 
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Tables  27  and  28  display  the  findings  from  the  evaluation  of  the  local  nationals  found  along  the 
route.  There  were  only  eight  local  nationals  and  somewhat  of  a  ceiling  effect  was  found. 
Despite  this  ceiling  effect,  significantly  fewer  local  nationals  were  found  when  the  AR  map  was 
used. 


Table  27.  Measure  15.  Number  of  local  nationals  found  (out  of  eight). 


Map  condition 

Mean 

SD 

2D-NU 

7.88 

0.94 

2D-R 

7.88 

0.91 

3PP 

7.59 

1.16 

AR 

7.28 

1.25 

F(3,93)  =  4.087,  p  =  0.009, ;/  =  0.116 

Table  28.  Paired  samples  test  for  Measure  15. 


Pair 

t 

df 

obtained  p 

required  p 

2D-NU  vs.  2D-R 

<1 

31 

0.99 

0.050 

2D-NU  vs.  3PP 

1.55 

31 

0.13 

0.017 

2D-NU  vs.  AR 

2.83a 

31 

0.01 

0.008 

2D-R  vs.  3PP 

1.72 

31 

0.1 

0.013 

2D-R  vs.  AR 

2.1T 

31 

0.01 

0.010 

3PP  vs.  AR 

1.22 

31 

0.23 

0.025 

“  p  <0  .05,  2-tailed. 


3.5  NASA-TLX  Results 

Table  29  and  figure  7  show  the  mean  ratings  on  the  NASA-TLX  subscales  and  total  workload 
scale  for  each  of  the  map  conditions. 

Table  29.  NASA  TLX  means  and  SDs  (in  parentheses). 


Scale 

2D-NU 

2D-R 

3PP 

AR 

Mental 

37.8  (20.6) 

39.1  (22.4) 

45.6  (22.8) 

53.6  (21.1) 

Physical 

19.7(16.9) 

20.8  (19.9) 

21.6(18.1) 

27.0  (24.4) 

Temporal 

34.2(21.6) 

35.8  (24.8) 

35.2  (21.2) 

47.2  (25.5) 

Performance 

34.8  (28.6) 

35.3  (29.3) 

41.9  (25.0) 

54.1  (23.4) 

Effort 

35.6  (22.0) 

36.6  (22.3) 

46.6  (21.5) 

56.3  (24.0) 

Frustration 

23.6  (20.2) 

26.1  (23.0) 

35.0  (23.3) 

56.4  (29.4) 

Total  workload 

36.3  (19.0) 

36.6(19.8) 

43.9(18.9) 

55.0(18.8) 

19 


60.0 

50.0 

40.0 

30.0 

20.0 

10.0 

0.0 


■  III 

J  1 

J  ■  1  J 

rwr 

i  pi 

■  North  up 

■  Rotate 

■  3rd  PP 

■  Aug  Real 


Figure  7.  NASA  TLX  means. 


Repeated  measures  ANOVA  (table  30)  indicate  that  there  were  significant  differences  among  the 
means  on  every  scale  except  physical  workload. 

Table  30.  Summary  of  NASA-TLX  ANOVAs. 


Scale 

F 

df 

P 

2 

n  p 

Mental 

8.1 a 

3,93 

<0.01 

0.21 

Physical 

2.29 

3,93 

0.08 

0.07 

Temporal 

4.24a 

3,93 

0.01 

0.12 

Performance 

5.1  la 

3,93 

<0.01 

0.14 

Effort 

10. 6a 

3,93 

<0.01 

0.256 

Frustration 

i7.r 

3,93 

<0.01 

0.355 

Total 

11. 2a 

3,93 

<0.01 

0.266 

><0.01 


Follow-on  paired  comparisons  were  conducted  using  Holm’s  Bonferroni  correction  for  family- 
wise  error  rates.  The  paired  comparisons  are  summarized  in  table  31. 
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Table  31.  Follow-on  paired  comparisons,  NASA-TLX. 


Mental: 


Pair 

t 

df 

obtained  p 

required  p 

NU  vs.  Rot 

<  1 

31 

NU  vs.  3rd  PP 

2.67a 

31 

0.012 

0.013 

NU  vs.  AR 

4.24“ 

31 

0.001 

0.008 

Rot  vs.  3rd  PP 

1.74 

31 

0.092 

0.025 

Rot  vs.  AR 

3.47a 

31 

0.002 

0.010 

3rd  PP  vs.  AR 

2.10 

31 

0.044 

0.017 

Temporal: 


Pair 

T 

df 

obtained  p 

required  p 

NU  vs.  Rot 

<  1 

31 

NU  vs.  3rd  PP 

<  1 

31 

NU  vs.  AR 

2.55 

31 

0.016 

0.010 

Rot  vs.  3rd  PP 

<  1 

31 

Rot  vs.  AR 

2.18 

31 

0.037 

0.013 

3rd  PP  vs.  AR 

2.77 

31 

0.009 

0.008 

Performance: 


Pair 

T 

df 

obtained  p 

required  p 

NU  vs.  Rot 

<  1 

31 

NU  vs.  3rd  PP 

1.57 

31 

0.126 

0.017 

NU  vs.  AR 

2.76 

31 

0.010 

0.008 

Rot  vs.  3rd  PP 

1.48 

31 

0.150 

0.025 

Rot  vs.  AR 

2.69 

31 

0.011 

0.010 

3rd  PP  vs.  AR 

2.21 

31 

0.034 

0.013 

Effort: 


Pair 

T 

df 

obtained  p 

required  p 

NU  vs.  Rot 

<  1 

31 

NU  vs.  3rd  PP 

2.83a 

31 

0.008 

0.013 

NU  vs.  AR 

4.14a 

31 

<  0.001 

0.010 

Rot  vs.  3rd  PP 

2.56a 

31 

0.015 

0.017 

Rot  vs.  AR 

4.25a 

31 

<  0.001 

0.008 

3rd  PP  vs.  AR 

2.22 

31 

0.034 

0.025 

Frustration: 


Pair 

t 

df 

obtained  p 

required  p 

NU  vs.  Rot 

<  1 

31 

NU  vs.  3rd  PP 

2.82a 

31 

0.008 

0.017 

NU  vs.  AR 

5.72a 

31 

<0  .001 

0.008 

Rot  vs.  3rd  PP 

2.33 

31 

0.033 

0.025 

Rot  vs.  AR 

4.48a 

31 

<  0.001 

0.010 

3rd  PP  vs.  AR 

3.96a 

31 

<  0.001 

0.013 

Total  workload: 


Pair 

T 

df 

obtained  p 

required  p 

NU  vs.  Rot 

<  1 

31 

NU  vs.  3rd  PP 

2.76a 

31 

0.010 

0.013 

NU  vs.  AR 

4.35a 

31 

<  0.001 

0.008 

Rot  vs.  3rd  PP 

2.29 

31 

0.029 

0.025 

Rot  vs.  AR 

3.93a 

31 

<  0.001 

0.010 

3rd  PP  vs.  AR 

2.73a 

31 

0.010 

0.017 

d  p  <  0.05,  2-tailed. 


The  2D-NU  and  2D-R  map  conditions  were  not  significantly  different  on  any  of  the  subscales  or 
on  total  workload.  The  workload  ratings  for  the  2D-NU  and  2D-R  maps  were  lower  than  the 
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ratings  for  AR  on  the  mental,  performance,  effort,  frustration,  and  total  workload  scales. 

Average  workload  ratings  for  the  2D-NU  map  were  lower  than  the  mean  ratings  for  3PP  map  on 
the  mental,  effort,  frustration,  and  total  workload  scales.  The  2D-R  and  3PP  maps  were 
significantly  different  on  the  effort,  frustration,  and  total  workload  scales  (the  2D-R  map 
workload  was  lower  on  all).  Mean  ratings  for  the  3PP  map  were  significantly  lower  than  the 
mean  AR  ratings  on  the  temporal,  effort,  fmstration,  and  total  workload  scales. 

In  summary,  workload  ratings  were  lowest  for  the  2D-NU  and  2D-R  map,  intermediate  for  the 
3PP  map,  and  highest  for  the  AR  map. 

3.6  Questionnaire  Results 

3.6.1  Post  Iteration  Questionnaire  Results 

The  2D-NU  map  was  rated  highest  for  keeping  track  of  one’s  orientation.  Some  Soldiers 
reported  difficulty  determining  in  which  direction  to  move  once  a  cardinal  direction  was  given 
by  the  platoon  leader  when  using  the  2D-R  map  because  they  had  to  first  determine  their  current 
orientation,  and  then  translate  it  in  their  heads.  However,  their  performance  times  did  not 
significantly  reflect  this  stated  difficulty  (see  the  2D-R  times  in  tables  10  and  13).  These 
Soldiers  also  reported  that  they  had  to  constantly  monitor  the  rotating  map  so  they  did  not 
become  confused  when  it  rotated.  Other  Soldiers  found  the  rotating  map  easy  to  use  and  very 
similar  to  the  map  in  their  GPSs. 

A  few  Soldiers  indicated  that  the  size  of  the  EUD  was  a  little  too  small  to  differentiate  between 
buildings  in  the  village  and  that  it  was  difficult  to  precisely  drop  a  chemlight  on  the  intended 
position  because  the  display  area  was  so  small  in  relation  to  the  size  of  their  fingers.  They 
suggested  that  they  needed  to  be  able  to  zoom  in  closer  to  accurately  place  the  chemlights  with 
their  “fat”  fingers.  They  reported  some  difficulty  with  placing  chemlights  on  areas  at  distances 
from  their  current  location  with  all  four  map  types;  however,  this  was  likely  an  artifact  from 
using  a  simulation,  because  many  of  the  distance  estimation  cues  available  in  a  natural 
environment  were  not  available  in  the  simulation  (i.e.,  accommodation  and  convergence  at  close 
distances,  binocular  disparity,  aerial  perspective,  etc.).  They  rated  accurately  placing  a 
chemlight  on  a  remote  location  as  being  more  difficult  with  the  3PP  and  AR  maps  than  with  the 
2-D  maps.  Soldiers  reported  that  the  chemlight  “shifted”  in  the  3PP  map  mode  and  that  it 
seemed  to  fall  in  a  different  location  than  they  intended.  Dropping  a  chemlight  in  an  accurate 
location  was  also  reported  to  be  difficult  in  the  AR  map  mode.  One  Soldier  suggested  that  it 
would  be  beneficial  to  use  the  crosshairs  on  the  eyepiece  to  accurately  place  chemlights. 

Estimating  distance,  following  a  pre-drawn  route,  and  the  projected  ease  of  planning  a  route  was 
rated  as  more  difficult  with  the  AR  map  than  with  the  other  three  map  types.  One  Soldier 
suggested  adding  two  indicators  in  the  corner  of  the  AR  map  screen  would  help  with  following  a 
pre-drawn  route.  The  first  would  always  point  in  the  direction  of  the  route  and  the  other  would 
point  to  the  waypoint.  These  would  help  the  Soldier  return  to  the  route  if  the  Soldier  got  off  the 
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route  and  could  not  find  it  and  orient  toward  the  waypoint.  Some  Soldiers  complained  that  the 
“lollipop”  stick  building  markers,  which  were  used  in  the  AR  mode,  did  not  actually  mark  the 
buildings  that  were  intended.  Others  complained  that  the  screen  became  so  cluttered  with 
markers  that  it  was  difficult  to  determine  what  was  being  marked,  see  all  the  information  on  the 
screen,  and  add  markers  in  the  same  area.  One  suggestion  was  to  make  the  markers  translucent 
and  allow  the  user  to  change  their  sizes  if  needed. 

Detailed  comments  from  the  post  iteration  questionnaire  can  be  found  in  appendix  B. 

3.6.2  End  of  Experiment  Questionnaire  Results 

The  end  of  experiment  questionnaire  asked  the  Soldiers  to  choose  the  map  they  preferred  for 
several  map-based  tasks.  Table  32  displays  their  choices. 

Table  32.  Number  of  Soldiers  choosing  each  map  type  as  best  for  map-based  tasks. 


Tasks 

Number  of  Responses3 

2D-NU 

2D-R 

3PP 

AR 

Best  for  dropping  a  chemlight  on  a  given  grid  coordinate 

19 

11 

1 

2 

Best  for  dropping  a  chemlight  on  the  map  to  show  the  location  of  an 
object  off  your  path 

14 

13 

2 

3 

Best  for  estimating  distance 

15 

10 

6 

2 

Best  for  following  a  pre-drawn  route 

17 

12 

2 

3 

Best  for  keeping  track  of  your  orientation 

24 

6 

2 

1 

Best  for  determining  which  building  you  are  facing 

17 

8 

5 

2 

Best  for  determining  the  best  way  to  go  around  an  obstacle  on  the  route 

20 

7 

0 

2 

Best  for  finding  the  location  of  a  chemlight  that  is  marked  on  the  map 

14 

13 

3 

6 

Most  intuitive  (i.e.,  easy  to  use/understand  without  much  training) 

22 

10 

1 

0 

Best  for  ease  of  engaging  touch  options 

22 

11 

2 

1 

Best  for  ease  of  enlarging  map 

19 

14 

3 

7 

Best  for  ease  of  moving  map  in  north,  south,  east,  or  west  directions 

19 

13 

2 

1 

Best  for  ease  of  zooming  in  and  out 

21 

14 

0 

0 

“Total  number  of  responses  may  exceed  N  as  some  participants  chose  more  than  one  map. 


While  the  2D-NU  map  was  chosen  as  best  by  the  most  Soldiers  for  each  map-based  task,  some  of 
the  Soldiers  preferred  other  map  types  for  the  majority  of  the  tasks  and  many  other  Soldiers 
chose  different  types  of  maps,  depending  on  the  task.  For  example,  some  of  the  Soldiers 
preferred  a  rotating  map  so  that  they  did  not  have  to  mentally  rotate  the  2D-NU  in  their  heads. 
One  Soldier  commented  that  the  3PP  map  would  have  been  more  useful  in  mountainous  terrain. 
Another  pointed  out  that  a  combination  of  one  of  the  2-D  maps  with  the  AR  would  work  well 
because  with  an  AR  map  it  was  easier  to  identify  buildings/objects  but  a  2-D  map  works  well  in 
providing  an  overall  view  of  the  area. 
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Many  Soldiers  saw  potential  in  the  AR  mode  but  felt  that  more  work  is  needed  to  make  it 
practical  to  use.  Overall,  the  2D-NU  appeared  to  create  the  lightest  cognitive  load.  It  made  it 
easy  to  keep  track  of  orientation  and  made  it  easy  to  determine  the  orientation  of  fixed  objects 
such  as  buildings. 

Table  33  lists  some  of  the  most  notable  suggested  improvements  to  an  EUD  map. 

Table  33.  Suggested  improvements  to  the  map  systems. 


Suggestion 

No.  of  Responses 

Make  the  AR  map  building  markers  transparent  or  make  them  only  pop  up  on 

2 

demand. 

Provide  the  ability  to  highlight  or  point  to  objects  on  screen  and  compute  my 

2 

distance  from  the  point. 

Provide  a  small  stylus  for  more  precision. 

2 

Provide  more  resolution. 

2 

Enlarge  the  screen. 

1 

Provide  the  ability  to  zoom  in  more  closely. 

1 

Provide  zoom  in  the  AR  mode. 

1 

Provide  real-time  imagery  so  that  current  barriers  and  changes  can  be  seen. 

1 

Provide  location  from  which  shots  have  been  fired. 

1 

Provide  location  and  direction  of  enemy  movement. 

1 

Decrease  lag  time. 

1 

Provide  the  ability  to  drop  a  symbol  at  current  location. 

1 

Detailed  responses  to  the  end  of  experiment  questionnaire  are  available  in  appendix  C. 

3.6.3  Experiment  Limitations  and  Experimenter  Observations 

3.6.3. 1  Route  Planning  and  Movement  in  the  Simulation 

An  important  function  of  a  map  is  to  give  the  Soldier  necessary  information  to  plan  a  route  for 
dismounted  movement.  Typical  military  maps  portray  information  related  to  distance,  contour, 
natural  obstacles,  and  manmade  structures.  Before  movement,  a  Soldier  in  the  field  would 
routinely  perform  a  map  reconnaissance  and  select  routes  (primary  and  alternate)  that  would 
provide  the  necessary  speed,  cover  and  concealment,  and  security  as  required  by  the  mission. 

We  anticipated  that  the  Soldiers  would  find  that  the  four  Information  Portrayal  Levels  (2D-NU, 
2D-R,  3PP,  and  AR)  would  provide  a  spectrum  of  capability  with  respect  to  route  planning. 
However,  in  planning  the  execution  of  this  experiment  in  the  virtual  simulation,  we  removed  this 
responsibility  from  the  Soldier.  To  control  the  pace  of  the  experiment  and  allow  for  comparisons 
of  movement  times  between  the  four  different  Information  Portrayal  Levels,  we  provided  a 
preplanned  route  in  the  operational  graphics  loaded  on  the  Soldier’s  EUD.  Therefore,  all  the 
Soldier  was  required  to  do  was  grab  the  joystick  and  move  along  the  route  without  thought  to 
proper  route  planning  considerations.  A  limitation  of  the  simulation  was  that  there  was  no 
penalty  to  the  Soldier  for  making  bad  navigational  choices.  The  Soldier  could  walk  through 
trees,  walk  backwards  for  extended  periods,  and  move  with  no  regard  for  security.  A  more 
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realistic  approach  would  be  to  conduct  a  similar  experiment  live  in  the  field  and  allow  (require) 
the  Soldier  to  plan  a  reasonable  route  prior  to  movement. 

Another  artifact  of  movement  in  the  simulation,  versus  in  a  live  environment,  was  the  speed  at 
which  the  Soldiers  were  able  to  move.  Originally,  the  scenarios  were  planned  to  last  about  1  h 
each  from  the  start  point  to  the  completion  of  the  scenario  in  the  MOUT  site.  In  practice,  each 
scenario  only  lasted  about  15  min.  This  was  likely  due  to  a  last-minute  software  change  in  the 
joystick  driver.  Soldiers  were  able  to  move  through  the  scenario  too  quickly.  This  caused 
elapsed  times  for  the  measures  of  performance  to  be  less  distinctive  from  each  other.  Again,  the 
next  best  experiment  would  be  to  conduct  a  similar  event  in  the  field  where  realistic  movement 
rates  would  be  imposed  by  the  natural  terrain. 

3. 6. 3. 2  No  Established  Standard  for  Time  or  Accuracy 

It  is  important  to  note  that  there  was  no  established  standard  for  each  of  the  performance 
measures  involving  time  and  chemlight  location  accuracy.  The  results  for  each  Information 
Portrayal  Level  are  intended  to  be  compared  with  each  other.  An  interesting,  but  not  necessarily 
essential,  experiment  would  be  to  run  similar  scenarios  in  the  field  with  legacy  systems,  i.e., 
map,  compass,  and  precision  lightweight  GPS  receiver  (PLGR),  for  example,  and  establish 
baseline  standards  for  each  of  the  measures.  Interested  stakeholders  could  then  assess  these  new 
Information  Portrayal  Levels  against  systems  currently  in  the  field. 

3. 6. 3. 3  Augmented  Reality  in  the  Standalone  Mode 

Lor  the  purpose  of  this  experiment,  the  Soldiers  were  required  to  use  only  one  Information 
Portrayal  Level  per  scenario  to  allow  for  comparisons.  This  is  an  unlikely  operational  restriction. 
When  such  systems  are  actually  fielded  to  Soldiers,  it  is  most  likely  that  they  will  be  able  to 
switch  back  and  forth  between  modes  to  use  the  mode  most  suitable  to  their  current  operational 
situation.  Alternately,  they  may  have  multiple  systems  that  give  them  more  than  one  capability 
at  the  same  time.  This  is  most  true  for  the  AR  mode.  As  run  in  this  experiment,  the  Soldier  had 
no  access  to  a  map  of  any  kind  during  the  scenario  using  the  AR  mode.  The  Soldier’s  only 
terrain  knowledge  came  from  what  Soldiers  could  see  in  their  field  of  view  on  the  video  monitor. 
The  operational  graphics  associated  with  that  scenario  were  only  visible  to  the  Soldier  when  the 
Soldier  maneuvered  into  the  vicinity  of  the  graphic.  Operationally,  a  Soldier  responsible  for 
movement,  planning  routes,  and  reacting  to  operational  graphics  would  always  have  an 
accompanying  map,  either  paper  or  digital,  with  the  necessary  information  available  for  viewing. 
So,  another  way  to  assess  with  AR  mode  would  be  to  allow  the  Soldier  capability  to  switch  back 
and  forth  from  AR  mode  to  one  of  the  map  modes,  or  provide  an  additional  map  that  could  be 
referenced  when  necessary.  This  should  be  a  planning  consideration  for  subsequent  studies. 

3. 6. 3.4  Chemlight  Task 

We  encountered  a  perplexing  phenomenon  during  the  experiment.  Each  scenario  followed  a 
written  script  that  was  managed  by  an  Observer  Controller  (OC)  sitting  one-on-one,  elbow-to- 
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elbow  with  each  Soldier.  The  OC,  acting  as  the  Soldier’s  company  commander,  would 
communicate  tactical  instructions  from  the  detailed  script  to  the  Soldier  to  control  the  action. 

The  OC  told  the  Soldier  when  to  move,  when  to  report,  what  to  report,  and  where  to  drop  what 
type  and  color  chemlights.  Upon  collecting  the  data  from  the  EUD  at  the  end  of  the  scenario,  we 
found  numerous  errors  in  the  color  and  type  of  chemlights  that  were  dropped  in  the  execution  of 
each  task.  There  was  no  penalty  assessed  for  the  incorrect  chemlight  type  or  color.  Based  on 
location  and  sequence  in  the  script,  we  could  determine  the  correct  chemlight  type  and  color  and 
assign  the  correct  value  for  assessment  of  the  performance  measure.  We  recommend  that  in 
subsequent  studies  that  we  grade  the  Soldiers  after  one  or  more  practice  scenarios  and  we  impose 
a  penalty  for  not  following  the  commander’s  instructions  precisely. 


4.  Discussion  and  Recommendations 


Soldiers  found  some  difficulty  in  using  the  small  (4.27-in)  touch  screen  on  the  smart  phone. 

Their  fingers  were  too  large  to  accurately  drop  the  chemlights  in  the  intended  location,  especially 
in  high  density  areas  such  as  McKenna.  Three  approaches  or  combinations  of  approaches  to  fix 
this  problem  should  be  investigated.  The  first  approach  is  to  allow  them  to  zoom  farther  in  so 
that  the  area  covered  by  their  fingers  is  smaller.  The  second  approach  is  to  provide  a  stylus  with 
a  fine  point  so  that  it  covers  a  smaller  area  than  the  fingers.  The  stylus  would  have  to  be  attached 
via  a  lanyard  so  that  it  does  not  become  lost.  The  third  approach  is  to  put  the  software  on  a  tablet 
with  a  display  size  of  7  to  10  in,  rather  than  a  phone  so  that  the  entire  scene  can  be  easily  seen 
without  zooming  and  so  that  the  finger  covers  a  smaller  area.  However,  this  approach  is  not 
without  cost  because  it  has  adverse  implications  on  the  weight  and  bulk  of  the  Soldiers’  loads. 

Findings  from  this  experiment  did  not  support  Wickens  and  Prevett’s  (1995)  model  of  aviation 
navigation  in  terms  of  the  egocentric  (2D-R)  display  providing  better  support  for  local  guidance 
and  Wicken’s  (1999)  evidence  for  performance  costs  associated  with  navigational  tasks  with  an 
exocentric  viewpoint  (i.e.,  mental  rotation  of  a  north-up  map  when  heading  south).  In  fact,  one 
of  the  tasks  which  Soldiers  reported  to  require  mental  rotation  was  trying  to  determine  the  north 
comer  of  the  building  when  using  the  2D-R  map  because  they  had  to  determine  their  heading 
direction  from  the  phone  and  then  mentally  rotate  the  building  to  find  which  corner  was  north. 
However,  these  findings  might  have  been  affected  by  the  scenarios  used.  Most  of  the  travel  was 
in  the  north,  northeast,  and  northwest  directions  and  did  not  require  mental  rotation  of  the  north 
up  map.  Thus,  a  decision  not  to  use  the  2D-R  map  might  be  premature.  Also,  although  the 
rotating  2D-R  map  was  only  preferred  by  9  of  32  Soldiers,  most  of  today’s  automotive 
navigational  maps  can  rotate  with  the  vehicle.  As  automotive  navigational  maps  become  more 
common,  more  Soldiers  of  the  future  may  enter  the  Army  with  a  high  degree  of  familiarity  with 
this  rotating  map,  and  may  be  less  inclined  toward  the  fixed  north  up  map. 


26 


Haskell  and  Wickens  (1993)  found  that  2-D  displays  and  perspective  displays  had  different 
strengths  and  weaknesses  for  aircraft  navigation,  and  this  may  be  the  case  for  land  navigation. 
The  perspective  display  fostered  superior  lateral  and  altitude  flight -path  tracking  accuracy,  and 
the  2-D  display  fostered  superior  airspeed  tracking  accuracy.  Because  the  terrain  used  in  this 
simulation  was  relatively  flat,  the  possible  strengths  of  the  3PP  display  were  not  examined  in  this 
experiment.  Soldiers  entering  unfamiliar  territory  must  rely  on  the  map  display  to  build  spatial 
awareness.  A  digital  2-D  map  that  uses  satellite  imagery  does  not  provide  contour  lines  so  that 
they  can  reconstruct  the  3-D  nature  of  the  area  in  which  they  will  operate.  Perspective  displays 
can  depict  all  three  dimensions  of  the  space,  without  requiring  Soldiers  to  integrate  textual  and 
spatial  information  like  they  have  to  if  they  use  maps  with  contour  lines.  It  may  be  that  Soldiers 
only  want  to  look  at  a  3PP  displays  for  specific  tasks  and  it  might  be  important  to  provide  the 
3PP  display  for  use  only  on  these  tasks. 

The  labels  on  the  AR  map  created  high  clutter  in  the  McKenna  area.  Suggested  improvements  to 
this  problem  include  making  the  label  markers  transparent,  having  the  markers  show  only  on  the 
fixed  objects  (buildings)  that  can  be  seen  with  line  of  sight  and  requiring  the  Soldier  to  request 
markers  for  objects  not  seen  in  line  of  sight,  or  even  a  combination  of  these  suggestions.  The  AR 
labels  in  this  simulation  were  not  always  on  the  correct  objects.  Geospatial  registration  accuracy 
must  be  high  for  an  AR  map  to  be  effective.  Labels  that  are  off  by  even  1  m  can  cause  confusion 
or  worse.  Also,  it  became  apparent  during  this  experimentation  that  AR  augments  a  map  but 
does  not  replace  it  because  it  does  not  provide  adequate  global  awareness  of  the  area.  However, 
further  experimentation  is  needed  before  taking  AR  out  of  consideration.  The  use  of  the  AR  map 
in  a  simulation  may  have  created  more  problems  than  would  be  experienced  in  a  real-world 
situation.  Holding  the  smart  phone  up  to  the  eye  to  see  the  simulation  through  the  phone’s 
camera  was  difficult  and  placing  chemlights  in  the  AR  mode  was  extremely  difficult  with  this 
method 


5.  Conclusions 


In  this  experiment,  the  2D-NU  map  was  preferred  by  the  most  Soldiers.  However,  generally 
their  performance  with  the  2D-R  and  the  3PP  maps  was  not  significantly  worse  than  with  the  2D- 
NU.  There  was  no  significant  difference  in  self-reported  cognitive  workload  ratings  between  the 
2D-NU  and  the  2D-R.  The  2D-NU  and  2D-R  map  mental  workload  ratings  were  significantly 
lower  than  those  of  the  AR  map,  and  the  2D-NU  map  mental  workload  ratings  were  significantly 
lower  than  for  the  3PP  map.  All  four  maps  types  offered  certain  advantages,  depending  on  the 
situation  and  terrain.  They  should  be  further  examined  in  a  live  experiment  that  contains  terrain 
that  could  demonstrate  their  advantages  before  they  are  removed  from  consideration.  It  is 
possible  that  all  four  map  types  should  be  included  on  an  EUD  so  that  Soldiers  can  choose  the 
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map  based  upon  the  situation  and  their  preferences  or  so  that  they  can  move  between  maps  when 
the  situation  warrants. 

There  were  three  limitations  in  this  experiment  that  were  based  upon  the  terrain  and  the 
simulation  that  may  have  adversely  affected  opinions  and  performance  with  the  2D-R,  3PP,  and 
AR  maps.  First,  the  terrain  around  the  McKenna  site  was  not  conducive  to  approaches  from  the 
north.  Approaches  from  the  north  would  have  required  the  Soldiers  to  use  more  of  their 
cognitive  resources  to  mentally  rotate  the  2D-NU  map  and  show  the  advantage  of  the  2D-R  map. 
For  example,  if  Soldiers  were  traveling  north  and  wanted  to  turn  east  it  would  be  easy  to  see  that 
a  right  turn  was  needed.  However,  if  Soldiers  were  traveling  south  and  wanted  to  turn  east,  they 
would  have  to  mentally  rotate  the  map  to  determine  that  they  need  to  turn  left.  Second,  the 
terrain  used  was  relatively  flat  and  did  not  show  the  advantage  the  3PP  map  may  have 
experienced  in  mountainous  or  hilly  terrain.  Third,  use  of  the  AR  map  in  a  simulation  may  have 
created  more  problems  than  would  be  experienced  in  a  real-world  situation.  This  experiment 
was  only  a  first  step  in  examining  the  four  map  types.  It  is  recommended  that  all  four  map  types 
be  examined  further  in  a  live  environment  that  does  not  contain  the  limitations  mentioned  above. 
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Appendix  A.  Demographic  Information 


DEMOGRAPHICS 
SAMPLE  SIZE  =  32 


MOS 

RANK 

DUTY  POSITION 

AGE 

09S  -  22 

15W  -  1 

OCS-31 

Officer  Candidates  -  31 

26  years 

1  IB  —  1 
13D  -  1 
13F  -  1 
15R-  1 

36B  -  1 
37F-  1 

NR -3 

E6-1 

Instructor  -  1 

(mean) 

1.  What  is  your  height?  71  inches  (mean)  -  Range  is  66  to  80  inches. 

2.  What  is  your  weight?  177  pounds  (mean)  -  Range  is  138  to  240  pounds. 

3.  How  long  have  you  served  in  the  military?  19  months  (mean) 

4.  Have  you  been  deployed  in  a  hostile  fire  zone?  25  No  _5_Yes  2  NR 

If  yes,  how  many  deployments  to  each  combat  area? 

Bosnia  _0^  #  of  deployments  0  months 

Iraq  6  #  of  deployments  78  months 

Afghanistan  1  #  of  deployments  12  months 

5.  With  which  hand  do  you  most  often  write?  28  Right  _4_  Left 

6.  With  which  hand  do  you  most  often  fire  a  weapon?  27  Right  _5_  Left 

7.  a.  Do  you  wear  prescription  lenses?  20  No  12  Yes 
b.  If  yes,  which?  _5_  Glasses  _6_  Contacts  J_  Both 

8.  Which  of  the  following  military  training  have  you  received? 
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Number  of  Responses 

32 

Basic  Training 

1 

One  Station  Unit  Training 

5 

Warrior  Leader’s  course 

2 

Airborne 

1 

Advance  Leader’s  course 

0 

Senior  Leader’s  course 

0 

Ranger 

12 

Combat  Life  Saver 

0 

Master  Gunner 

5 

Other:  armorer,  landscaping 

9.  Using  the  scale  below,  rate  your  level  of  experience  with  the  following  computer  software  and 
computer  related  activities. 


1 

2 

3 

4 

5 

6 

7 

No 

experience 

Below 

Average 

Slightly 

below 

average 

Average 

Slightly  above 
average 

Above 

average 

Expert 

MEAN 

Microsoft  Windows  98,  2000,  XP,  etc. 

5.29 

Computer  based  games 

4.69 

Internet 

5.38 

AKO 

4.06 

My  Pay 

4.13 

Smart  phone 

4.91 

Army  digital  systems  (e.g.  FBCB2) 

1.81 

I  would  self  rate  my  computer  skills  as: 

4.97 

10.  Using  the  scale  below  rate  your  level  of  experience  with  the  following  map  based  devices 
and  location  aids. 


1 

2 

3 

4 

5 

6 

7 

No 

experience 

Below 

Average 

Slightly 

below 

average 

Average 

Slightly  above 
average 

Above 

average 

Expert 

MEAN 

Military  topographic  maps 

4.09 

Civilian  paper  maps 

4.47 

Civilian  Global  Positioning  Systems  (GPSs) 

4.84 

Military  GPSs  (PLGR,  etc.) 

2.06 

Google  Earth 

4.47 

Computer  based  mapping  products  (Mapquest,  etc.) 

4.91 

Military  Grid  Reference  System 

3.28 

Latitude/Longitude  system 

3.50 

Knowledge  of  infantry  tactics,  techniques,  and  procedures  (TTP) 

2.81 

Smartphone  based  maps 

3.97 

Other: 

3.25 
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Appendix  B.  Post  Iteration  Questionnaire  Results 


POST  ITERATION 
SAMPLE  SIZE  =  32 


Maps 

2D  North  up  map 

2D-NU 

2D  Rotating  map 

2D-R 

3rd  Person  perspective  map 

3PP 

Augmented  reality  map 

AR 

1.  Using  the  scale  below,  please  rate  the  map  that  you  just  used  for  the  following  tasks. 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


TASKS 

MEAN 

2D-NU 

2D-R 

3PP 

AR 

Dropping  a  chemlight  on  a  given  grid  coordinate 

5.85 

5.90 

4.27 

4.00 

Dropping  a  chemlight  on  the  map  to  show  the  location  of 
an  object  off  your  path 

5.91 

5.78 

4.31 

4.00 

Estimating  distance 

5.09 

5.00 

4.94 

3.78 

Following  a  pre-drawn  route 

6.19 

6.22 

5.97 

4.41 

Keeping  track  of  your  orientation 

6.31 

5.28 

5.44 

4.00 

Determining  which  building  you  are  facing 

5.50 

5.56 

5.19 

3.78 

Determining  the  best  way  to  go  around  an  obstacle  on  the 
route 

5.19 

5.03 

4.62 

4.16 

Finding  the  location  of  a  chemlight  that  is  marked  on  the 
map 

5.77 

5.60 

5.47 

4.65 

Ease  of  engaging  touch  options 

6.32 

6.06 

4.90 

5.48 

Ease  of  enlarging  map 

6.41 

6.16 

5.16 

4.61 

Ease  of  moving  map  in  north,  south,  east,  or  west 
directions 

6.41 

6.22 

5.53 

4.90 

Ease  of  zooming  in  and  out 

6.35 

6.09 

5.29 

4.67 

Planning  a  route  over  a  kilometer  (km)  long  in  flat  terrain 

5.26 

5.29 

5.26 

4.32 

Planning  a  route  less  than  100  meters  (m)  long  in  flat 
terrain 

5.33 

5.45 

5.23 

4.70 

Planning  a  route  over  a  km  long  in  hilly  and  mountainous 
terrain 

5.10 

4.81 

5.00 

4.24 

Planning  a  route  less  than  100m  long  in  hilly  and 
mountainous  terrain 

5.00 

5.05 

5.00 

4.29 
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Comments 


No.  of  Responses 


2D-NU 

Excellent.  1 

Great!  1 

Easy  to  use.  1 

By  far,  my  favorite.  Simple,  organized,  and  easy  to  follow.  Note:  Dropping  chem  1 
lights  was  much  easier  after  practice. 

Easy  to  navigate  using  this.  You  always  know  where  you  are  and  where  you  are  1 
heading. 

It  was  extremely  easy  deciding  what  direction  you  are  facing.  This  made  it  easy  to  1 
see  what  direction  you  needed  to  go  for  the  next  objective. 

Easy  to  figure  out  the  buildings  based  on  looking  around  and  comparing  to  map.  1 

A  lot  easier  to  orient  which  direction  you  are  going  in.  2 

Very  easy  to  use  and  adjust  in  the  2D  north  up  view  setting.  1 

Fixed  north  orientation  made  it  easier  to  maintain  spatial  awareness.  1 

I‘m  becoming  better  at  using  the  system  in  general.  1 

I  like  the  satellite  picture  to  follow  over  a  non-easy  topo  paper  map.  Call  for  fire  1 
would  be  awesome  with  terrain  location  ability  of  the  phone. 

Very  easy  to  locate  points  in  relation  to  N  and  much  less  of  a  headache  than  AR.  1 

Compared  to  the  AR,  this  was  a  breeze.  Felt  like  I  was  using  a  paper  map  and  1 

compass  without  the  hassle  of  either. 

Since  the  compass  did  not  move,  it  made  it  simpler  and  easier  to  follow.  1 

Good  for  maneuvering,  not  so  good  for  micro  identification.  1 

Smart  phone  was  easy  to  follow  and  this  view  it  seemed  to  be  easy  to  drop  chem  1 
lights  when  I  wanted  them. 

Hard  to  place  chem  lights  on  exact  location.  If  I  could  use  the  smart  phone  to  place  1 
a  point  on  the  terrain  in  front  of  me  (laser/crosshatch),  could  it  more  easily 
correspond  to  a  digital  point  on  the  smart  phone  display? 

Dropping  chem  lights  is  fairly  easy  but  mistakes  can  easily  be  made.  Luckily  they  1 
are  easily  fixed. 

Dropping  a  chem  light  between  buildings  can  be  difficult  if  the  display  area  is  too  1 
small  for  your  finger. 

Difficult  to  make  symbol  (chem  light)  locations  in  this  mode  and  know  if  they  were  1 
placed  on  the  actual  target. 

The  only  real  problem  I  had  was  the  placement  of  markers  for  objects  that  were  not  1 
at  my  present  location.  If  the  object  was  off  some  distance  from  my  position,  I  felt 
uncertain  in  regard  to  placement  accuracy. 

Difficult  to  recognize  which  building  is  which  in  the  simulator  screen.  1 

Difficult  to  always  match  up  buildings  on  map  to  screen.  1 

Determining  which  building  I  was  at  was  a  little  difficult  because  my  location  was  1 
off  on  the  map. 

Icon  and  location  of  the  site  did  not  match.  It  was  hard  to  move  around  obstacles.  1 

The  blue  chevron  on  the  phone  was  slightly  off  which  made  it  slightly  difficult  for  1 

map-to-video  correlation. 
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Comments 


No.  of  Responses 


Placement  of  chevron  was  off  on  map  to  relation  of  character.  1 

Map  to  location  chevron  and  simulator  was  slightly  difficult  to  coordinate.  1 

Estimating  distance  from  current  location  to  target  was  difficult. 

The  GPS  was  slightly  off  making  some  of  the  directions  difficult  to  follow  in  areas  1 
with  smaller  avenues  of  motion.  The  map  showed  me  walking  on  top  of  buildings 
instead  of  beside  them  as  I  saw  on  the  screen. 

Does  not  zoom  in/out  far  enough  to  my  liking.  1 

2D-R 

Overall,  a  great  tool.  3 

The  display  overall  is  great  and  easy  to  use.  3 

Very  useful  and  much  more  efficient  than  standard  use  of  map  land  navigation.  1 

So  far  it  is  the  easiest  way  to  navigate  on  the  map.  Not  difficult  to  find  a  comer  of  a  1 
building. 

Easy  to  orientate  self  to  map  and  move  in  directed  direction  on  this  map.  1 

The  rotation  of  the  map  was  great  and  touching  the  screen  worked  excellent.  1 

Easier  and  faster  when  finding  direction.  2 

Size  of  icons  is  just  right.  1 

This  map  is  very  effective  in  keeping  your  bearings  with  your  surroundings,  but  1 

causes  a  little  confusion  when  a  specific  direction  (N  S  E  W)  is  requested. 

Trouble  maintaining  orientation  when  directions  (NSE  W)  were  given.  Distances  1 
were  difficult  to  gauge  without  a  scale. 

Choosing  to  drop  an  icon  on  the  map  is  easy;  however,  placement  accuracy  is  more  1 
difficult  for  individuals  with  larger  fingers. 

Sometimes  found  it  hard  to  orient  which  direction  I  was  going  in.  Maybe  if  the  1 

compass  icon  was  more  detailed  it  would  be  easier. 

At  one  point  getting  around  an  obstacle,  I  went  the  wrong  direction  on  the  route;  had  1 
to  double  back  and  reorient  myself. 

If  the  blue  chevron  could  constantly  stay  centered,  that  would  make  the  guidance  1 
system  perfect. 

It  was  difficult  to  determine  obstacles  until  you  are  on  top.  1 

It  was  a  bit  difficult  to  determine  distance.  1 

No  way  to  judge  distance  when  you  zoom  in/out.  Scale  for  boxes?  1 

A  bit  of  clarity  would  help  movement.  1 

Could  add  a  button  to  drop  symbols  or  chem  lights  on  your  exact  location  instead  of  1 
taping  the  map  screen  and  possibly  adding  the  symbol  to  the  map  in  the  wrong 
location. 

The  dropping  of  the  chem  lights  is  not  as  precise  as  it  could  be.  1 

Coordination  was  a  bit  difficult  due  to  the  lag  and  moving  north.  The  moving  north  1 
doesn’t  seem  to  be  helpful  in  this  particular  scenario. 

When  given  instructions  or  descriptions  in  cardinal  directions,  it  definitely  took  a  1 
little  longer  to  translate  that  into  a  place  on  the  map.  Unsure  of  the  benefit  of  the 
rotation  map/N  symbol. 

I  didn’t  like  the  map  rotating  as  I  rotated.  It  was  hard  to  estimate  distance.  1 
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Comments 


No.  of  Responses 


The  rotation  of  the  map  on  the  phone  makes  it  difficult  to  locate  yourself  at  a  fast  1 
pace. 

The  rotating  map  was  a  little  difficult.  It  made  it  harder  to  orient  which  direction  to  1 
move  unless  you  were  constantly  monitoring  it. 

Letting  objects  rotate  around  me  makes  pathfinding  tougher  when  referencing  the  1 
map. 

Knowing  which  building  is  which  is  confusing  when  moving  and  spinning  on  the  1 
fixed  axis.  Constantly  having  to  track  N  is  mildly  touchy  but  not  difficult  except 
when  you  need  to  approach  from  a  specific/side. 

3PP 

Excellent;  3D  is  very  useful.  1 

Easiest  to  use  out  of  all  other  programs.  1 

The  map  is  well  down  and  easy  to  use.  1 

The  map  was  effective  for  depth  perception.  1 

Easier  to  identify  and  move  towards  icons  on  3D  rather  than  AR.  1 

Easy  to  correct  when  you  got  off  course  since  direction  and  path  were  highlighted.  1 

Labeling  where  you  had  been  was  a  great  idea.  Having  the  WP  with  ascending 
numbers  was  also  a  great  indicator  of  direction. 

It  seemed  that  as  I  tried  to  move  the  map  north  or  south,  it  zoomed  in  at  the  same  1 
time. 

Being  able  to  use  touch  screen  worked  great,  but  where  I  was  placing  markers  and  1 
where  they  showed  up  were  off. 

Lag  time  can  take  a  little  getting  used  to.  1 

Placement  of  symbols  (i.e.,  chem  lights)  can  be  difficult  due  to  size  of  fingers  1 

causing  actual  placement  to  be  off. 

Dropping  chem  lights  in  exact  location  difficult  due  to  large  fingers.  Have  a  pen-  1 
like  selector  would  allow  more  accurate  location  drops. 

Placing  items  was  very  difficult  to  perform.  Objects  seldom  appeared  where  I  1 

thought  I  was  pressing. 

Dropping  chem  lights  could  be  better.  2 

Overall,  easy  to  navigate,  but  at  times  was  difficult  to  put  chem  light  in  exact  desired  1 
spot  on  map. 

Dropping  chem  lights  on  target  only  once  was  very  rare.  Had  to  move  and  re-drop  1 
almost  every  point. 

Accurately  dropping  chem  lights  was  very  difficult  in  this  mode.  They  seemed  to  2 
fall  very  far  from  where  I  intended  them  to  go. 

Very  difficult  to  drop  a  chem  light  in  the  correct  spot.  The  map  didn’t  follow  me;  I  2 
had  to  move  the  map  to  keep  myself  centered  in  it. 

Dropping  a  chem  light  on  a  point  was  difficult  the  chem  light  was  always  shifting.  1 
Determining  when  I  was  on  a  waypoint  was  difficult  if  I  wasn’t  directly  on  the 
path. 

Trying  to  very  precisely  drop  chem  lights  can  be  a  little  challenging  on  the  touch  1 
screen.  Perhaps  a  pen  or  pin  device  could  be  used  in  order  to  precisely  drop  chem 
lights. 
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Comments 


No.  of  Responses 


It’s  difficult  to  precisely  place  chem  lights  because  one  has  to  guess  a  little  lower  1 
than  one  would  think.  I  really  did  not  see  a  benefit  with  the  3D  perspective. 

I  tried  to  change  the  color  of  a  chem  light  I  dropped,  but  was  unable  to  access  the  1 
options  menu,  even  with  the  map  fully  zoomed  in. 

3D  was  not  very  good.  I  think  2D  view  was  a  lot  easier  and  user  friendly.  1 

This  type  of  map  took  getting  used  to.  The  first  four  markers  needed  moving  around  1 
to  get  placed  right.  After  getting  the  hang  of  it,  I  found  it  easy  as  the  other  maps. 
Marker  placement  on  movement  route  map  was  difficult  due  to  inaccurate  1 

placement.  Placement  on  village  map  was  not  an  issue.  Tracking  buildings  edges 
due  to  shadows  in  image  was  difficult.  Image  shadows  were  on  the  south  side  of 
the  building. 

It  was  hard  to  determine  which  side  of  the  building  I  was  looking  for.  1 

Obstacles  not  on  map.  Maps  would  have  to  be  super  up-to-date  to  help  navigate  1 

around  some  obstacles. 

When  I  tried  to  zoom  in  on  the  map,  it  would  not  zoom  any  more.  Would  have  been  1 
nice  to  zoom  slightly  further. 

Hill  and  mountainous  terrain  not  present  in  scenario.  Obstacles  not  present  on  map,  1 
unable  to  judge  distances  and  location  of  C-wire  until  moving  character  on  screen 
and  moving  to  location  obstacle. 

AR 

Once  on  the  path,  following  it  was  relatively  easy.  1 

The  touch  screen  for  the  map  is  really  good.  1 

Finding  places  described  in  cardinal  directions  relative  to  known  points  was  harder  1 
than  any  other  mode. 

The  only  trouble  I  found  was  finding  the  building  #  in  the  AR  mode  on  its  “lollipop”  1 
stick.  I  double  checked  numerous  times  different  colors  that  match  physical  copy 
of  map  would  help. 

Building  markers  did  not  actually  mark  buildings.  1 

Building  names  should  “float”  on  left  or  right  side  as  they  pass  out  of  view.  Should  1 
also  be  able  to  select  bldg  to  highlight  against  background  (would  create  third 
arrow  on  mipmap). 

The  reality  map  was  more  challenging  since  on  a  small  screen  as  opposed  to  the  1 

human  eye  sight  and  how  much  you  can  see. 

Once  you  place  certain  number  of  icons  on  the  map,  it  becomes  harder  to  mark  any  1 
other  targets  in  that  same  area. 

The  “N”  on  the  paper  map  was  pointing  in  a  different  direction  compared  to  “N”  on  1 
Smartphone. 

If  you  get  off  the  route,  I  found  it  hard  to  navigate  back  on  to.  2 

Place  route  on  ground?  1 

The  route  lines  blurred  often  and  were  difficult,  if  not  impossible,  to  find  once  you  1 
walked  a  certain  distance. 

This  method  was  extremely  difficult  with  blind  spots  and  finding  targets.  1 

It  is  slightly  difficult  to  drop  chem  lights  at  specific  locations  because  you  have  to  be  2 
a  certain  distance  away. 
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Comments 


No.  of  Responses 


Placement  of  chem  light/markers  is  difficult.  Would  rather  have  a  “drop”  chem  light  1 
at  my  location.  Estimate  distance  to  target  almost  impossible  if  focused  only  on 
eyepiece.  Icons  are  so  large  that  they  block  the  view.  Information 
overload/frustration.  Spent  most  time  looking  at  eyepiece. 

Difficult  to  make  symbol  (i.e.,  chem  lights)  locations  in  this  mode  and  know  if  they  1 
were  placed  on  the  actual  target. 

Difficult  to  accurately  drop  a  chem  light  at  the  intersection  of  the  object  and  ground.  1 

The  chem  lights  and  markers  were  very  difficult  to  see  around/through.  2 

Marker  size  was  very  disorienting.  1 

The  chem  light  markers  fill  the  screen  when  they  are  approached,  making  it  1 

impossible  to  see  other  things.  So  some  level  of  transparency  would  make  it  easier 
to  navigate,  while  still  demonstrating  the  nearness  of  the  item  (chem  light, 
waypoint,  etc.). 

Map  icons  get  in  the  way  of  AR  map  display  you  do  not  adjust  in  size.  1 

The  location  markers  were  a  bit  too  large.  1 

Had  to  back  track  in  order  to  place  chem  lights.  1 

Too  many  symbols  on  the  screen.  Hard  to  differentiate  between  chem  lights  and  1 
markers.  Difficult  to  place  chem  lights  where  they  belong  on  the  screen. 

It  would  be  great  if  we  could  use  the  crosshairs  to  look  at  (aim  at)  the  location  to  1 
more  accurately  place  markers. 

Need  indicator  in  corner  that  always  points  toward  route  and  another  that  points  to  1 
waypoint. 

This  would  be  better  if  the  user  had  control  with  the  markers  sizes  and  opaque  (see  1 
through)  values.  I  think  it  would  be  helpful  if  the  chem  light  markers  would  lay 
down  on  the  ground  until  the  user  stops  walking  or  something  to  that  nature. 

When  chem  lights  are  dropped,  I  keep  overshooting  the  targets.  Use  of  a  stylus  1 

could  be  helpful. 

In  the  small  display  in  the  bottom  corner  it  would  have  made  it  easier  to  orient  1 

locations  and  checkpoints  had  it  displayed  features  and  points  on  it. 

Much  more  complicated  and  somewhat  confusing.  1 
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2.  Using  the  scale  below,  please  evaluate  the  following  features  of  this  interface. 

1  2  3  4  5  6  7 

Extremely  bad  Very  bad  Bad  Neutral  Good  Very  good  Extremely  good 


FEATURES 

MEAN 

2D-NU 

2D-R 

3PP 

AR 

Size  of  symbols  (e.g.,  chemlight) 

5.91 

5.84 

5.48 

3.53 

Size  of  symbols  as  you  zoom  in  and  out  of  map 

5.88 

5.87 

5.45 

3.95 

Size  of  text 

6.00 

5.87 

5.29 

4.29 

Size  of  grid  lines 

6.03 

5.93 

5.61 

4.75 

Size  of  the  overall  map 

6.28 

6.16 

5.68 

4.83 

Map  colors 

6.44 

6.42 

6.26 

5.84 

Placement  of  text 

5.97 

5.77 

5.48 

4.47 

Comments  No.  of  Responses 


2N 

Excellent.  1 

Easiest  to  use  for  this  specific  simulation.  1 

Very  useful  and  easy  to  use.  1 

Great  graphics,  easy  to  find  an  object.  No  problems  marking  an  object.  1 

The  text,  colors,  zoom  in/out  work  perfect  on  this  map  and  make  navigating  very  1 

easy. 

All  sizes  were  good.  2 

No  issues  with  map  or  symbols  with  this  map.  1 

Did  not  obstruct  map,  unlike  AR.  1 

Great  job  on  the  interface.  I  had  no  trouble  understanding  what  I  was  looking  at.  1 

Easy  to  follow  and  discern  information  provided,  configuration  of  chevron  to  map  2 

offset. 

More  realistic  due  to  the  fact  that  when  you  are  navigating  outside,  you  are  not  1 

rotating  map  as  you  were  in  other  map  configurations  that  were  tested. 

Speed  wise,  Smartphone  was  able  to  keep  up  with  actual  speed  of  moving.  1 

Was  able  to  focus  much  more  on  the  actual  screen  and  not  the  phone.  1 

Less  strain  on  eyes  and  simpler  to  orient  oneself.  1 

The  size  of  the  chem  symbols  on  the  map  is  hard  to  land  just  right  with  big  fingers.  1 
You  could  only  zoom  in  so  close  to  be  accurate.  Would  be  nice  to  zoom  in  a  little 
closer  to  better  drop  chem  lights. 

I  should  have  zoomed  in  closer  to  see  a  more  accurate  depiction  of  where  I  was  in  1 
relation  to  WPs. 

Zoom  intervals  too  large.  1 

Problems  seeing  text  on  top  of  buildings  without  zooming  in  quite  a  bit.  Problems  1 
with  placement  of  text  on  buildings. 

I  would  prefer  that  symbols  (chem  lights,  etc.)  were  a  little  smaller.  1 

The  route  lines  could  be  a  little  smaller  so  you  could  still  see  them  on  the  map  but  it  1 
wouldn’t  cover  more  terrain  image  than  necessary. 
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Comments 


No.  of  Responses 


2R 

Excellent.  1 

I  like  the  system  and  the  application.  1 

The  interface  is  easy  to  use  and  well  done.  2 

All  are  good.  1 

So  far,  easiest  to  use.  2 

Could  really  make  a  life  saving  application  for  soldiers  on  the  battlefield  and  make  1 
maneuvering  a  lot  more  efficient. 

Practical.  1 

The  placement  and  allowing  the  map  to  move  is  great.  1 

It  was  easy  to  see  beyond  the  symbols  to  see  what  was  actually  on  the  map.  1 

Easy  to  follow.  1 

Easy  to  read.  1 

Maps  were  really  good  and  clear.  1 

Map  is  easy  to  read  because  it  is  like  the  GPS  systems  of  our  Smart  phones.  1 

Text  is  very  easy  to  read  on  the  screen.  1 

Anyone  who  has  used  a  cell  phone,  GPS  or  a  regular  one  could  do  it  with  no  1 

problem.  It  is  actually  easier  to  use. 

Much  better  than  3D.  1 

Easy  to  discern  amongst  targets  and  locations.  1 

Size  of  some  symbols  can  be  reduced.  1 

For  me  this  rotation  is  more  difficult  to  orient  to  without  testing  longer  and  learning.  1 
I  would  not  get  rid  of  this  option  because  it  is  useful  to  other  people. 

Sometimes  you  need  to  stand  by  and  wait  until  Smartphone  map  location  will  keep  1 
up  with  your  actual  location. 

Only  complaint  is  that  the  chevron  goes  off  the  map  when  you’re  moving.  1 

Not  always  easy  to  verify  where  chevron  was  on  map  in  relation  to  screen.  1 

Placement  of  text  to  ID  buildings  is  difficult  to  use  when  reorienting  to  move  to  a  1 
specific  side  of  the  building.  The  ID  factor  for  the  building  is  the  orientation  of  the 
characters  so  when  they  move  constantly,  I  feel  like  south  is  moving  unless  I  check 
where  north  is  in  the  comer. 

Building  markers  were  difficult  to  identify  on  map.  1 

3D 

Excellent.  1 

The  maps  are  really  easy  to  follow.  1 

The  colors  and  text  of  map  are  great.  1 

Map  on  phone  easy  to  read,  clear  lines,  not  cluttered.  1 

Works  well  with  the  3D  view  to  help  see  the  distance  of  objects  marked.  1 

The  interface  is  well  done  and  easy  to  use.  After  little  practice  it  is  almost  natural.  1 
Easier  to  navigate  through  city  and  finding  building  numbers.  1 

Given  it’s  a  Smartphone,  I’m  not  sure  how  much  bigger  the  map  can  be;  but  it  did  1 
feel  small  at  times. 

Maps  do  not  account  for  obstacles  and  the  best  way  to  maneuver  around  them.  1 
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This  sky  view  takes  getting  used  to.  It  was  not  as  accurate  to  me  as  the  straight  1 

overhead  view. 

Symbols  made  the  screen  cluttered  at  times.  1 

Size  of  casualty  and  friendly  forces  need  to  be  smaller;  they  are  oversized  and  1 

overwhelm  the  visual. 

Placement  and  size  of  text  in  the  city  to  identify  buildings  was  difficult.  Darkened  1 
shadows  obscure  the  text. 

Difficult  to  see  the  building  identifiers  when  I  was  in  the  village.  They  blended  in  1 
with  the  buildings. 

The  waypoint  labels  should  be  a  higher  contrast  color  so  that  they  stand  out  more.  1 
Black  labels  were  difficult  to  see  on  the  darker  map. 

Angle  view  distorts  and  overlaps  text.  1 

AR 

The  text  was  great!  1 

I  like  the  size  of  the  symbols  and  of  the  grid  lines,  but  sometimes  they  can  block  out  1 
your  vision  entirely. 

The  symbols  on  the  map  all  clustered  together.  2 

Symbols  too  large.  1 

Symbol  size  was  too  big.  You  could  barely  see  the  text  on  the  actual  map.  It  was  1 
hard  to  understand  the  distance  between  placed  icons  on  the  screen.  Hard  to 
determine  which  building  is  which. 

Symbols  are  too  big.  The  writing  on  the  ground  is  hard  to  read  unless  you  are  right  1 
on  top  of  the  writing. 

The  size  of  the  symbols  were  very  large  and  obscuring  other  targets  and  visuals.  2 

Symbols  should  be  transparent  so  that  you  could  see  through  them.  They  are  also  1 

slightly  too  large. 

The  symbols  on  the  map  were  extremely  large  to  start  and  were  unable  to  tell  when  1 
one  was  passed.  Once  size  was  reduced,  the  size  was  much  easier  to  see  and 
follow. 

I  used  the  printout  map  to  navigate  more  than  the  phone.  1 

Buildings  should  have  labels  float  on  them.  Should  be  in  high  contrast  color  -  not  1 
black.  Need  to  be  able  to  set  “groups”  of  chems  and  minimize  unnecessary 
groups.  Should  be  able  to  shrink  icon  size  or  placement. 

Building  numbers/labels  were  not  on  buildings.  2 

Difficult  to  tell  what  waypoint  I  was  at.  I  did  not  realize  the  number  was  written  on  1 
the  ground  and  even  then  it  was  difficult  to  read. 

Difficult  to  read  the  text  on  symbols.  Some  symbols  (casualty,  friendly)  were  1 

missing. 

Text  hard  to  read  when  parallel  with  ground  (black  WP  hard  to  see  at  distance).  1 

Difficult  to  identify  bldg  #s  when  close  or  around  multiple  bldgs,  in  that  mode 
only. 

The  interface  is  crowded.  It  also  is  a  bit  challenging  to  stay  on  the  proper  path  if  you  1 
deviate  at  all. 

When  in  the  actual  urban  environment,  buildings  were  very  difficult  to  identify.  1 
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Comments  No.  of  Responses 

With  a  higher  number  of  chem  lights  in  the  area,  it  becomes  very  difficult  to  1 

navigate  at  all  due  to  the  markers  blocking  view. 

Hard  to  place  chem  lights  where  I  wanted.  1 

Need  to  have  the  chem  light  option  menu  first  as  opposed  to  scrolling  down  to  add  1 
new  system. 

I  was  so  distracted  by  the  map  that  I  had  a  terrible  time  following  or  finding  the  1 

actual  target.  As  it  is,  I  would  not  want  this  in  real  world  application.  If  the  user 
could  adjust  setting  to  personality,  I  think  this  would  be  a  great  product. 

Only  trouble  I  found  was  when  a  target  (i.e.,  wounded  soldier)  was  hidden  behind  1 
medic-cross  symbol. 

3.  Overall  comments  on  this  map  interface: 

Comments  No.  of  Responses 

2D-NU 

Excellent,  north  being  fixed  is  great.  1 

Best  map  I  used  today. 

Very  good.  1 

This  program  is  awesome  overall.  1 

Overall,  user  friendly.  3 

Overall  easy  to  use  and  navigate.  2 

Overall  successful.  1 

Much  easier  to  use  than  the  AR  method.  1 

Practical.  1 

Very  neatly  laid  out  display  that  makes  it  easier  to  stay  oriented.  1 

It  would  be  easy  to  use  this  system  to  move  from  one  grid  location  to  another  or  to  1 

follow  a  preset  route. 

Very  simple  interface;  makes  it  easy  to  shoot  azimuths  and  find  a  general  direction.  1 
The  steady  N  also  helps  to  keep  our  bearings. 

I  preferred  the  fixed  north  orientation  to  the  rotating  map.  It  made  it  much  easier  to  1 
maintain  spatial  awareness. 

Locating  a  specific  side  of  a  building  (i.e.,  west,  east,  etc.)  was  easier  in  this  map  1 
than  in  the  2D-R  map. 

It  was  easy  to  use  and  correct  when  mistakes  were  made.  1 

This  interface  is  easier  to  navigate  through  urban  terrain,  but  easily  lost  on  track  with  1 
no  path. 

Everything  about  this  interface  was  intuitive  and  simple.  The  only  problem  was  1 

marker  placement,  and  that  was  not  very  significant  if  I  was  attempting  to  place  a 
marker  at  my  location. 

Estimating  distance  to  WPCs  in  relation  to  WP  lines  was  frustrating.  Distance  to  1 
enemy  hostiles  is  difficult  to  mark  on  the  map. 

Distance  judging  is  hard  but  that  may  stem  from  the  simulation  more  so  than  the  1 

interface. 
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Comments 


No.  of  Responses 


More  difficult  to  follow  the  route  and  move  to  waypoints  and  other  objectives  in  this  1 
map  than  in  the  2D-R  map.  In  other  words,  2D-R  was  easier. 

Difficult  to  estimate  distances  and  difficult  to  place  chem  lights  exactly.  1 

I  think  that  multiple  options  should  be  kept  for  style  and  use  of  every  personality.  I  1 
personally  like  the  overview  2D-NU  where  other  people  will  like  the  rotation. 

Keep  all  of  it. 

The  chevron  being  slightly  off  made  moving  around  the  MOUT  site  difficult.  1 

Location  of  chevron  and  orientation  not  completely  in  sync.  1 

Only  issue  was  with  location  of  chevron  on  map.  1 

N  up  map  can  be  a  little  confusing  when  making  multiple  turns;  compared  to  the  1 
rotating  map. 

There  was  some  small  lag  in  time  which  almost  made  me  cross  over  my  paths  before  1 
I  was  ordered  to. 

Some  difficulty  with  the  lag  between  the  screen  and  the  map  caused  me  to  overrun  1 
the  waypoints  in  some  cases. 

Icons  did  not  correspond  with  the  actual  locations  and  it  was  somewhat  hard  to  1 

gauge  the  distance  throughout  the  course. 

Had  a  little  trouble  at  some  points  with  the  map  orientation  when  navigating  around  1 
buildings. 

Building  identification  was  hard  due  to  text/bldg  resolution.  Map  locations  with  1 
multiple  items  become  muddled. 

2D-R 

Awesome  map  interface.  3 

The  map  interface  was  the  easiest  to  determine  orientation  and  to  navigate  through  1 
the  urban  environment. 

Great.  1 

Easy  to  use.  6 

Intuitive.  1 

Easy  to  follow  lines.  1 

Chem  lights  are  easy  to  place  and  track.  1 

Very  easy  to  follow  the  routes  and  place  things  on  the  touch  screen.  1 

Easy  to  navigate  and  maneuver.  2 

Easy  to  read.  1 

Easy  to  follow.  1 

So  long  as  you  are  always  aware  that  your  map  orients  around  you,  this  was  by  far  1 
the  most  enjoyable  method  of  navigation. 

The  rotation  compass  actually  makes  it  easier  to  find  sides  of  buildings  (for  me,  1 

anyway). 

Very  easy  to  follow  paths  and  locate  items,  but  when  trying  to  envision  the  relative  1 
placement  of  objects  in  N  E  W  S  directions,  a  moment  of  mental  readjustment  is 
required. 

If  the  blue  chevron  could  stay  centered,  that  would  be  ideal.  1 

The  distance  is  hard  to  judge.  2 

Map  is  not  good  for  navigating  around  obstacle.  1 
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Comments 


No.  of  Responses 


Effective  for  all  facets  except  for  specific  direction  requests.  1 

Easy  to  drop  items.  1 

I  noticed  it  took  a  little  more  mental  thought  locating  sides  of  buildings  when  the  N  1 
arrow  was  not  fixed. 

Text  on  buildings,  adding  compass  reference  rather  than  fixed  north.  Arrow  tells  you  1 
which  direction  you  are  headed  not  where  north  is  for  making  longer  distances. 
Keeping  yourself  oriented  was  a  little  difficult  on  the  rotating  map.  2 

The  changing  orientation  of  north  made  it  difficult  to  maintain  spatial  awareness.  1 
The  green  return  box  is  helpful  when  I  traveled  off  the  map. 

One  problem  with  this  orientation  is  that  one  can  confuse  which  direction  to  go  on  1 
the  route.  In  other  words,  they  can  be  turned  around.  However,  this  may  seem 
more  problematic  in  a  virtual  environment  and  it  may  be  a  no  issue  in  the  real 
world. 

The  zoom  in  and  out  function  is  a  little  difficult  to  use  and  I  didn’t  like  that  I  had  to  1 
re-center  the  map  on  the  chevron  as  I  was  moving. 

Placing  chem  lights  in  an  accurate  location  is  a  little  difficult  with  large  fingers.  1 

As  the  map  changes,  you  must  mentally  redraw  your  position  relative  to  the  route.  1 
A  fixed  perspective  allows  you  to  pathfind  to  and  from  your  point  of  departure 
without  known  waypoints  changing  position. 

3D 

Overall,  an  excellent  tool.  Very  impressed.  2 

Good  overall.  1 

Easy  to  use  and  well  done.  3 

Overall,  much  easier  to  navigate  throughout  simulation.  1 

Easy  to  locate  and  mark  targets.  1 

Fixing  mistakes  is  easy  and  best  of  all  the  map  and  interface  make  sense.  1 

Easy  to  pick  up  after  only  a  few  minutes  once  lag  is  decreased.  This  could  definitely  1 
make  land  navigation  a  breeze.  May  cause  a  little  confusion  with  map  always 
adjusting  around  soldier’s  location.  It’s  nice  to  be  certain  that  the  top  of  the  map  is 
always  north. 

3D  map  is  really  good  for  estimating  distance.  1 

Extremely  easy  for  direction  (N,E,S,W),  distance  and  location  of  buildings  that  had  1 
coordinates/labels  attached  to  them. 

This  view  is  easier  to  follow  on  a  route  than  the  AR  view;  but  judging  distance  1 

(while  marking  things  from  afar)  is  difficult. 

In  general  I  had  no  issues  navigating  on  the  map,  except  for  finding  a  particular  1 

corner  of  a  building. 

The  map  mode  was  good  for  obtaining  an  overall  picture  of  the  area,  but  I  found  it  1 
harder  to  follow  than  some  of  the  other  map  types. 

Easier  to  visualize  the  town  in  3D,  but  more  difficult  to  follow  the  path.  There  was  1 
significant  lag  that  made  it  hard  to  stop  on  waypoints  and  get  back  on  the  path. 

It  is  very  applicable  but  I  still  prefer  the  2D  map  with  north  up.  1 
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Comments 


No.  of  Responses 


Didn’t  see  a  way  to  go  around  obstacles  on  map  (this  question  appeared  in  the  1 

previous  questions),  and  the  lag  updating  points  as  you  changed  direction  could  be 
slightly  improved. 

This  map  interface  was  more  difficult  in  general.  2 

Had  a  hard  time  dropping  chem  lights  and  other  items  in  the  specific  location.  1 

The  navigation  wasn’t  as  much  of  a  problem  as  the  placement  of  chem  lights.  1 

Placement  felt  inaccurate  and  took  multiple  attempts  to  get  in  correct  location. 

Had  trouble  placing  chem  lights  exactly  where  I  want  them.  Much  harder  to  do  in  1 
3D  mode. 

Once  a  chem  light  was  dropped,  it  was  difficult  to  select  and  move  that  chem  light.  1 

It  seemed  the  blue  arrow  that  was  my  symbol  was  too  close  to  the  bottom  of  the  1 

screen  a  lot  of  the  time.  That  made  it  difficult  to  drop  chem  lights  at  my  location 
because  I  would  have  to  watch  where  I  wanted. 

The  placement  of  the  map  is  difficult  to  get  oriented  to  by  following  route  from  map  1 
on  phone. 

It  was  easy  to  forget  to  watch  the  monitor  and  focus  only  on  the  phone.  1 

I  think  it  would  be  easier  to  not  use  this  method.  1 

Not  sure  the  3D  map  really  gives  an  advantage  over  the  2D  map,  and  may  have  1 

several  disadvantages  (higher  hardware  requirements  for  computing,  less  accurate 
chem  light  drops,  etc.). 

Could  use  the  ability  to  zoom  in  more  for  accurate  placement  of  symbols  (i.e.,  chem  1 
lights)  on  the  map. 

The  3D  view  is  harder  to  use  as  far  as  zoom  and  trying  to  see  where  you  are  on  the  1 
map. 

Zooming  issues  even  at  max  zoom  unless  you  move  the  map  so  that  the  chevron  is  1 
directly  in  the  bottom  center,  the  map  looks  smaller.  Perspective  distance  at  the 
viewing  angle. 

Must  be  more  aware  to  your  surroundings.  1 

Harder  to  find  markers  due  to  the  angle.  1 

Colors  for  some  labels  could  be  made  brighter.  1 

AR 

Great  graphics  overall.  1 

Was  fun.  1 

Good  for  distance  and  up  close.  1 

Easy  to  maneuver  in  the  area.  1 

I  like  the  idea  and  think  it  will  be  very  useful.  1 

Makes  land  nav  less  stressful  knowing  you  have  this  technology  (i.e.,  finding  1 

direction  very  quickly). 

Like  the  route  lines  and  WP  to  find  objective.  1 

It’s  ok,  but  it  will  take  some  time  to  get  used  to.  1 

Simply,  the  objects  need  to  be  a  bit  smaller  to  make  it  worth  using;  otherwise  it  1 

worked  great. 

The  interface  was  too  crowded,  but  it  is  easy  to  use  overall.  1 

I  believe  with  some  practice  it  would  be  easy  to  use.  1 
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No.  of  Responses 


Can  be  an  effective  tool  to  see  where  things  were  on  a  given  point,  but  it  is  difficult  1 
to  manage  while  navigating.  This  method  should  not  replace  general 
foreknowledge  of  the  AO,  better  as  a  supplement. 

Works  really  well  for  following  a  pre-arranged  route,  but  trying  to  look  at  1 

surroundings  becomes  difficult  with  all  the  labels  and  lines. 

Seems  to  have  been  some  sort  of  malfunction  and  it  made  the  map  almost  useless.  I  1 
can  see  the  advantages  of  the  system,  but  I  was  frustrated  with  my  experience. 

I  would  suggest  making  the  markers  or  symbols  on  the  map  transparent  in  order  that  2 
you  could  see  through  them  if  you  were  too  close.  This  would  still  allow  them  to 
be  visible  from  a  distance  and  up  close  without  blocking  your  vision. 

It  was  much  harder  to  orient  oneself  in  the  urban  environment.  Buildings  were  1 

difficult  to  identify,  and  I  was  nearly  totally  blinded  by  markers  in  the  augmented 
reality  view. 

Building  numbers  should  be  marked  more  clearly.  1 

Orientation  as  a  whole  is  difficult  to  follow  and  relate  to  mission  as  a  whole.  1 

The  lag  was  significant,  making  it  difficult  to  orient  yourself.  2 

Icons  need  to  be  translucent.  1 

AR  mode  needs  a  two  finger  mode  to  set  location/height.  This  would  prevent  1 

“higher”  presses  registering  as  “distant”  presses.  Pinch  to  move  selector  point 
closer  -  spread  to  increase  distance.  Overall,  AR  becomes  cluttered  and  confusing 
in  tow,  and  easily  lost  in  field. 

Make  symbols  user  friendly  with  setting  changes.  1 

Make  symbols  lay  down  until  the  user  requests  them  to  pop  up.  1 

Make  symbols  smaller  and  position  symbols  closer  to  targets.  1 

I  prefer  the  2D  views  for  navigating.  The  number  of  symbols  on  the  screen  is  1 

distracting.  It  was  difficult  to  know  when  I  reached  a  waypoint  or  the  release 
point. 

Markers  on  the  map  grew  larger  as  you  approached,  obstructing  your  vision  of  1 

waypoints,  med  points,  etc. 

Make  it  so  a  person  can  point  to  the  middle  of  the  screen  at  the  location  of  the  target  1 
point.  If  I  want  to  pick  the  point  I’m  standing,  point  phone  at  my  feet  then  select 
the  marker.  I  would  like  a  crosshair  and  laser  dot  on  the  screen  to  point  and  a 
button  to  click  that  would  place  the  target  point.  Like  a  camera  picture  merged 
with  a  crosshair  of  a  gun  and  shoot  their  feet. 

Takes  a  while  to  get  used  to  view  angle  and  placing  accurate  chem  lights  and  1 

standing  on  specific  point. 

Very  hard  to  place  chem  lights  on  targets  at  0°.  1 

The  chem  light  placement  was  frustrating,  as  was  the  size  of  the  symbol  on  the  1 

screen. 

It’s  difficult  to  determine  your  spatial  understanding  (distance  from  one  point  to  1 

another). 

I  ended  up  focusing  more  on  the  map,  which  was  woefully  inadequate  in  orienting  1 
my  location  and  showing  the  routes.  The  chem  light  placement  was  also 
extremely  difficult. 
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Comments 


No.  of  Responses 


A  little  bit  difficult  at  times  on  a  Smart  phone;  would  be  great  as  an  eyepiece  “heads-  1 
up  display”  as  noted  at  the  beginning. 

1 

More  confusing  and  not  as  user  friendly  as  the  other  systems;  would  confuse  many 
soldiers  I  fear. 

More  complex  and  feature  full  than  necessary  at  the  AR  level.  Floating  directional  1 
arrow  with  distance  to  point  counter  would  allow  for  better  navigation. 

Hard  to  navigate  left/right  while  moving.  1 

Can’t  distinguish  where  items  are.  1 

Would  like  to  drag-n-drop  icons;  icons  too  large;  become  visual  obstacles.  1 
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Intentionally  Left  Blank. 
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Appendix  C.  End  of  Experiment  Questionnaire  Results 


END  OF  EXPERIMENT 
SAMPLE  SIZE  (N)  =  32 


_ Map  Type _ 

2D  North  up  map  (2D-NU  ) _ 

2D  Rotating  map  (2D-R) 

3rd  Person  perspective  map  (3PP) 
Augmented  reality  map  (AR) 


1.  Which  map  type  would  be  best  for  the  following  tasks? 


Tasks 

Number  of  Responses* 

2D-NU 

2D-R 

3PP 

AR 

Dropping  a  chemlight  on  a  given  grid  coordinate 

19 

11 

1 

2 

Dropping  a  chemlight  on  the  map  to  show  the  location  of  an 
object  off  your  path 

14 

13 

2 

3 

Estimating  distance 

15 

10 

6 

2 

Following  a  pre-drawn  route 

17 

12 

2 

3 

Keeping  track  of  your  orientation 

24 

6 

2 

1 

Determining  which  building  you  are  facing 

17 

8 

5 

2 

Determining  the  best  way  to  go  around  an  obstacle  on  the  route 

20 

7 

0 

2 

Finding  the  location  of  a  chemlight  that  is  marked  on  the  map 

14 

13 

3 

6 

Which  map  is  most  intuitive  (i.e.,  easy  to  use/understand 
without  much  training)? 

22 

10 

1 

0 

Ease  of  engaging  touch  options 

22 

11 

2 

1 

Ease  of  enlarging  map 

19 

14 

3 

7 

Ease  of  moving  map  in  north,  south,  east,  or  west  directions 

19 

13 

2 

1 

Ease  of  zooming  in  and  out 

21 

14 

0 

0 

Hotal  number  of  responses  may  exceed  N  as  some  participants  chose  more  than  one  map 


Comments  No.  of  Responses 

Any  would  work  about  the  same  in  my  opinion.  1 

A  mixture  of  modes  would  work  best.  Different  modes  make  sense  at  different  1 

times  and  situations. 

2D-NU 

2D-NU  was  the  easiest  to  use,  and  fastest  interface.  8 

2D-NU  made  me  feel  confident  in  controlling  what  I  was  doing  and  was  simple  to  1 
figure  out. 
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Comments 


No.  of  Responses 


Simplicity  of  2D-NU  really  worked  well.  1 

Easiest  to  orient  my  direction  when  north  stayed  up  as  in  2D-NU  mode.  1 

I  was  able  to  maintain  my  spatial  awareness  and  navigate  in  the  town  with  2D-NU.  1 

2D-NU  would  be  my  choice.  1 

2D-NU  best  simulates  an  actual  map.  1 

2D-NU  map  never  rotated  which  let  your  mind  be  at  ease  while  moving  and  1 

receiving  orders. 

Although  other  map  types  would  be  helpful,  in  this  particular  scenario,  2D-NU  1 

seemed  to  be  the  most  helpful. 

2D-NU  and  2D-R  are  the  same  and  work  wonders.  1 

2D-NU  and  2D-R  were  very  similar  in  navigation.  Although  set  N”  made  it  easy  to 
find  any  required  object. 

Most  of  these  basic  tasks  would  be  easiest  in  either  the  2D-NU  or  2D-R  modes.  1 

2D-NU  and  2D-R  are  already  being  used  in  Smartphone’s  for  maps  and  directions.  It  1 
would  be  easiest  for  modern  soldiers  to  adapt  to  and  use  efficiently  in  the  field. 

2D-R 

2D-R  offers  one  of  the  best  interface  features  and  easiest  to  use.  3 

I  might  get  used  to  the  2D-R  if  I  had  more  practice.  1 

2D-R  was  the  map  that  took  the  shortest  amount  of  time  to  understand.  1 

2D-R  was  the  most  intuitive  for  me.  1 

2D-R  needs  to  be  able  to  quickly  and  easily  identify  the  N  S  E  W  side  of  a  building.  1 
2D-R  is  fine,  but  for  me  I  can  lose  my  sense  of  orientation  if  I  am  always  “moving  1 

up”  a  screen. 

3PP 

With  3PP  and  AR  I  felt  that  I  could  not  accurately  mark  targets.  1 

3PP  made  it  easier  to  estimate  distance  and  find  existing  chemlights  on  the  screen.  1 

3PP  was  pretty  useless.  1 

AR 

Some  benefits  to  marker  placement  accuracy  with  the  AR.  1 

With  more  training,  the  AR  could  be  more  helpful.  1 

AR  is  a  great  idea,  but  there  should  be  a  better  way  to  plot  information  like  a  1 

separate  small  screen  that  is  in  2D  and  maybe  handheld;  also  with  a  stylus  or 
something. 

I  like  the  potential  of  AR.  Needs  to  have  adjustment  ability  given  to  the  user  as  1 

stated  in  my  (AR)  evaluation. 

Maps  are  easy  to  use  and  follow.  Routes  on  AR  map  are  more  difficult  to  follow;  2 
routes  would  be  easier  to  follow  if  on  the  ground. 

AR  map  type  was  difficult  to  determine  whether  an  icon  is  behind  or  in  front  of  the  1 
building. 

The  AR  option  is  too  cluttered  right  now.  It  makes  it  challenging.  1 

The  AR  was  the  most  cumbersome  and  straining!  1 

The  AR  had  too  much  going  on  and  could  be  overwhelming.  1 

Downside  of  AR  was  large  symbols  and  poor  overlay  of  the  planned  route  to  follow;  1 
also  dropping  symbols  on  a  certain  grid. 


50 


2.  Which  map  type  was  best  for  the  following  features? 


Number  of  Responses 

Features 

2D- 

2D- 

3PP 

AR 

NR 

NU 

R 

Size  of  symbols  (e.g.,  chemlight) 

18 

12 

1 

0 

1 

Size  of  symbols  as  you  zoom  in/out  of  map 

21 

10 

0 

0 

1 

Size  of  text 

17 

12 

1 

0 

2 

Size  of  grid  lines 

16 

9 

4 

1 

2 

Size  of  the  overall  map 

16 

11 

1 

1 

3 

Map  colors 

19 

8 

2 

1 

2 

Placement  of  text 

17 

12 

1 

1 

1 

Best  overall  map 

20 

9 

1 

1 

1 

Comments  No.  of  Responses 

Map  colors  were  excellent  on  pretty  much  every  map  type.  1 

2D-NU  and  2D-R  maps  are  both  excellent.  2 

I  love  the  ease  of  the  2D-NU;  user  friendly.  3 

2D-NU  was  a  lot  less  confusing  looking  directly  down  at  targets  and  symbols/text  1 
was  not  obstructing  any  view. 

2D-NU  is  a  close  second,  but  for  the  first  time  around  I  thought  the  2D-R  was  much  1 
easier  to  use.  However,  I’m  sure  with  time  that  3PP  and  AR  would  become  easier 
to  use. 

2D-NU  was  by  far  the  best  in  terms  of  design  and  functionality.  It  is  easier  to  tell  a  1 
change  of  direction  by  looking  at  the  chevron  than  the  compass  arrow. 

I  think  the  optimum  system  would  be  a  combination  of  the  2D-NU  and  the  AR.  The  1 
AR  would  be  the  mode  you  are  in  most  frequently  but  the  2D-NU  would  exist  to 
give  you  a  birds  eye  perspective  and  more  of  an  overall  picture  of  the  area  you’re 
working  in. 

I  don’t  see  any  benefit  or  difference  in  2D-NU  and  2D-R;  it  would  come  down  to  1 
personal  opinion. 

I  did  not  notice  much  of  a  difference  in  size  and  color  for  2D-NU,  2D-R,  or  3PP.  1 

2D-R  map  took  a  little  getting  used  to,  but  once  accustomed,  it  was  by  far  the  best  1 

map  with  the  2D-NU  a  close  second. 

Response  time  of  rotating  map  (2D-R)  might  be  too  slow  to  be  useful;  most  soldiers  1 
probably  already  well  versed  in  N-oriented  maps. 

I  like  the  potential  of  AR.  Needs  to  have  adjustment  ability  given  to  the  user  as  1 

stated  in  my  (AR)  evaluation. 

AR  had  symbols  that  were  distracting  and  made  it  difficult  to  navigate.  1 

Symbols  were  quite  oversized  on  AR.  For  that  reason  it  was  hard  to  mark  an  object  1 
with  chemlight. 

AR  map  was  the  most  difficult  to  use  and  follow  routes  and  identify  buildings  when  1 
close. 

I  don’t  like  it  when  the  (2D-R)  map  rotates;  I  lose  sense  of  direction.  1 
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Comments 


No.  of  Responses 


2 
1 

3.  If  you  could  only  choose  one  type  of  map,  which  one  would  you  choose? 


Would  love  to  have  used  3PP  map  on  mountainous  terrain. 
I  would  prefer  symbols  to  be  a  bit  smaller. 


Map  Type 

Rank 

2D  North  up  map  (2D-NU  ) 

20 

2D  Rotating  map  (2D-R) 

10 

3  Person  perspective  map  (3PP) 

1 

Augmented  reality  map  (AR) 

1 

Comments  No.  of  Responses 

The  simpler  maps  seemed  more  practical  for  tactical  uses.  1 

I  love  the  ease  of  the  2D-NU  .  1 

2D-NU  is  easier  to  view  map  in  its  whole.  1 

2D-NU  is  the  most  versatile  for  completing  different  tasks.  1 

2D-NU  -  command  view;  2D-R  -  PL  view;  3PP  -  PSG  view;  and  AR  -  individual/  1 
simple  movement  view. 

Judging  distance  was  the  easiest  on  2D-NU  and  2D-R.  1 

2D-NU  and  2D-R  were  very  easy  and  understandable  for  use.  1 

2D-R  was  the  easiest  to  use  and  navigate.  1 

Using  rotating  (2D-R),  I’d  rather  it  tell  me  which  direction  I’m  going.  1 

2D-R  seemed  un-useful,  rather  than  helpful,  in  my  opinion.  1 

I  think  if  you  could  have  two  of  the  maps,  2D-NU  and  AR  would  be  the  best  option.  1 

AR  has  a  lot  of  potential,  but  in  these  instances  it  wasn’t  the  most  intuitive.  1 

AR  would  be  more  helpful  (if  adjusted  to  eyewear)  in  a  dangerous  urban  1 

environment. 

I  like  the  potential  of  AR.  Needs  to  have  adjustment  ability  given  to  the  user  as  1 

stated  in  my  (AR)  evaluation. 

AR  is  annoying  because  of  the  icons.  If  they  were  transparent  or  smaller,  it  would  1 
be  number  1. 

With  AR  the  symbols  filling  the  screen,  when  you  get  close  to  them,  obstructs  your  1 
view  of  everything  else,  including  the  other  symbols.  They  need  to  have  some  sort 
of  transparency  or  stacking  function  that  lets  you  see  what’s  behind  the  closes  one. 

AR  would  be  detrimental  to  me  if  I  was  trying  to  navigate.  2 

AR  needs  to  improve  lag  time,  which  I  know  is  not  the  software  but  the  hardware.  1 
3PP  is  more  productive  when  plotting.  1 

3PP  would  be  helpful  with  missions  on  hills,  mountains,  etc.  3 

3PP  would  probably  be  better  for  more  diverse  terrain.  1 

I  found  little  benefit  with  the  3PP  view.  1 

3PP  is  overkill.  With  the  other  options  we  have,  3PP  is  not  an  advantage  (in  my  1 

opinion). 
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Comments 


No.  of  Responses 


One  downside  of  all  4  map  types  is  that  you  have  to  stand  by  and  wait  until  chevron  1 
catches  up  with  you. 

4.  What  is  best  way  to  move  your  cursor  to  a  specific  grid  location? 


Number  of  Responses 

Manually  type  in  the  10-digit  coordinate 

8 

Use  a  “go  to”  dialog  box  to  choose  from  a  list  of 

plotted  entities  (e.g.,  PL  symbol)  to  move  to  their 

2 

grid  coordinate  on  the  map 

Touch  a  location  on  the  screen 

17 

Other  (specify  below) 

2 

NR 

3 

Comments  No.  of  Responses 

For  a  specific  grid,  it  will  be  better  to  type  it  in  manually  for  accuracy.  2 

Manually  would  take  more  time,  but  if  accuracy  is  the  key,  then  that  is  the  best  way. 

A  “go  to”  dialog  might  quickly  become  overwhelmed.  Imagine  scrolling  through  a  1 


list  3  miles  long  because  of  all  of  the  points  that  had  been  added. 

“Go  to”  if,  however,  there  are  a  lot  of  points  (6+),  it  would  be  cumbersome.  1 

Input  data  for  “where  to  go.”  1 

Check  grid  location  by  tapping.  1 

Best  way  is  to  place  finger  on  grid  to  point  it  out.  1 

Easier  to  use  touch  screen  to  get  an  approximate  location  of  an  object.  1 

Touch  is  the  fastest  and  easiest  way,  although  it  may  not  be  accurate.  2 


Touch,  but  then  I  have  to  drag  it  to  the  more  exact  location.  Chemlights  do  not  fall  1 
where  I  think  I’m  touching  the  screen.  Not  very  easy  to  move/edit  chemlights. 

Would  prefer  to  drag-n-drop  an  icon  with  a  crosshair  to  show  where  it  will  be 
placed. 

If  grid  plots  are  available  on  touch  screen,  this  would  likely  be  the  best  option.  1 

Love  the  google  earth  ability.  It  is  important  to  still  have  grid  coord  input  method  to  1 
find  a  specific  target  given  from  someone  else.  Scrolling  would  not  be  time 
efficient. 

Other:  a  stylus,  pin,  cursor,  or  mouse.  1 

5.  If  time  and  money  were  no  issue,  what  additional  features  would  you  like  this 
technology  to  provide?  (For  example,  are  there  any  features  that  you’ve  seen  in  games 
or  movies  that  you  would  like  to  have?) 

Comments  No.  of  Responses 

Simple  is  sometimes  more  effective  and  efficient.  Too  many  extras  slow  down  1 

system  and  clutter  screen. 
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Comments 


No.  of  Responses 


Highlight  objects  on  screen  and  know  my  distance  from  the  point.  1 

For  marked  buildings,  labeling  each  side  with  a  small  N  S  E  W  or  even  putting  a  N-  1 
up  symbol  on  top  of  the  building  would  make  it  a  lot  easier;  though  the  trade-off 
would  be  a  busier  map  (more  symbols,  etc.).  Also  I  think  the  ability  to  zoom  in 
more  closely  can  add  accuracy  and  clarity  for  the  symbols. 

Be  able  to  see  building  numbers  even  if  another  building  is  in  the  way.  1 

Features  like  seeing  through  building  and  see  people  from  far  away.  1 

I  was  able  to  play  with  an  eyepiece  monitor/camera  at  a  soldier  expo.  If  this  system  1 
were  integrated  into  that  system  it  would  be  great,  along  with  a  secondary  input 
device. 

More  real  time  information  displayed  in  the  HUD.  1 

A  real  time  IR  view  to  notice  potential  threats.  1 

Infrared.  1 

Maybe  someday  they  would  be  able  to  provide  live  time  maps  via  drone  planes  with  1 
cameras  in  order  to  give  the  most  current  high  resolution  over  head  image.  This 
could  help  with  things  like  barriers  that  you  might  not  see  on  a  map  if  the  map 
photo  was  taken  before  the  barrier  was  built.  Maybe  spy  planes  would  be  able  to 
link  up  directly  with  the  mapping  device. 

Maybe  higher  resolution  on  the  maps.  1 

Add  a  feature  on  the  iphone  that  places  a  point  on  the  terrain  in  viewing  in  AR  that  1 
will  identify  with  a  point  on  the  actual  map  display. 

It  would  be  good  if  you  could  touch  something  in  AR  mode  and  get  its  distance  from  1 
you. 

Spatial  two  finger  control  for  AR  (as  seen  in  all  space  games)  nav  and  target  arrows  1 
(most  air/space  fighter  games). 

Distance  markers  to  points,  the  distance  to  the  point  ahead  of  you  are  two  major  1 

terrain  features. 

Instead  of  just  dropping  one  chemlight  to  mark  the  wire,  be  able  to  mark  the  whole  1 
wire  so  soldiers  know  where  the  openings  in  the  wire  are  located. 

Zoom  features  on  the  AR.  The  targeting  of  the  hostiles  could  have  been  the  most  1 
accurate  in  the  AR  mode  if  you  could  just  zoom  in  enough  to  clearly  see  where 
they  stood. 

Change  icon  size  on  AR  map  types  and  have  them  indicate  the  exact  location  of  1 

nonmoving  objects  from  any  point  of  view. 

In  the  AR,  keep  the  orientation  of  the  radar  fixed  up  with  a  rotating  compass  on  the  1 
outer  ring  and  points  inside  adjusting  as  the  person  turns. 

In  the  AR,  if  the  orientation  in  the  bottom  comer  had  details  and  features  on  it;  it  1 
would  be  easier  to  use  and  locate  yourself  on  the  map  in  relation  to  what  you’re 
around. 

In  AR,  the  map  on  the  bottom  left  should  have  the  route  on  it  clearly  or  as  an  option.  1 
The  lag  confuses  my  location. 

Use  more  pixels  on  the  map.  1 

A  little  bit  bigger  screens.  1 

Speed  it  up.  1 
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Comments 


No.  of  Responses 


Crosshair  and  camera  click  targeting.  Everyone  on  this  system  would  have  a  nav  1 
point  over  them  like  Halo.  The  phone  could  display  orders  in  a  comer  or  overlay 
then  fade  back  to  the  map,  and  fade  back  last  orders  when  needed. 

Communication  technology  between  team  members;  voice  controls.  1 

A  way  to  see  enemy  would  help  if  possible.  1 

Enemy  locations  installed  onto  ballistic  eyewear  so  no  separate  device  is  required.  1 
Location  of  shots  fired  or  enemy  movement.  1 

Something  that  could  detect  thermal  readings  and/or  IED  signs  would  be  incredible.  1 
I  would  decrease  the  lag  time  and  also  get  better  satellite/topographic  images  stored  1 
on  the  phone. 

Perhaps  use  a  small  pen  in  order  to  more  precisely  place  drop  points.  Additionally,  1 
once  a  waypoint  is  reached,  it  would  fade  away  or  change  color  in  order  to  remind 
you  that  you  have  already  reached  that  waypoint. 

A  small  pen  of  stick  (possibly  attached  to  glove  or  wrist)  to  help  pinpoint  drop  1 

points  on  little  Smartphone  screen.  I  have  fat  fingers  so  my  chemlights  were  a  little 
off. 

Drop  symbol  at  current  location  button.  1 


6.  At  what  echelon  should  this  technology  be  used?  Check  all  that  apply. 


Number  of  Responses 

Individual  Soldier 

16 

Fire  team 

27 

Squad 

30 

Platoon 

27 

Company 

21 

Above  company  level 

10 

Comments  No.  of  Responses 

This  technology,  if  advanced  adequately,  should  be  used  by  every  soldier  who  goes  1 
outside  the  wire. 

I  think  if  you’re  going  to  have  a  system  like  this,  everyone  on  the  ground  should  1 
have  their  own  system  that  links  up  with  their  squad/platoon/  company,  etc. 

Everyone  could  be  able  to  input  things  in  the  system  but  maybe  you  could  turn  on 
layers  in  order  to  only  see  certain  people  and  what  they  have  marked  on  the  map. 

For  example,  if  you  are  back  at  headquarters  and  you  only  want  to  see  one 
individual  squad’s  movement,  you  could  turn  all  the  other  squads  off.  Leaders  at 
the  company  level  could  track  everyone’s  progress  and  continue  to  issue  specific 
tasks  to  different  platoons/squads/fire  teams. 

Would  give  great  situational  awareness  to  everyone  in  a  company.  1 
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Comments 


No.  of  Responses 


Platoon/squad/fire  team  leaders  because  they  are  the  most  involved  with  mission  1 
planning/implementation;  and  are  the  ones  directing  where  their  soldiers  are  going. 

Above  company  can  oversee;  it  would  be  the  most  beneficial  at  the  squad/  platoon  1 
level.  Easier  to  keep  organized. 

Would  be  most  effective  for  smaller  teams  and  squads,  but  I  could  see  it  becoming  1 
linked  together  for  the  higher  echelons. 

Everyone  would  know  where  everyone  is.  It  would  be  easier  to  coordinate  attacks,  1 
extractions;  everything  really. 

If  money  is  not  an  issue,  then  every  soldier  should  be  able  to  use  this  technology.  1 


Otherwise,  at  least  every  team  leader  should  have  it. 

Information  sharing,  call  for  fire  and  Medevac  applications  for  this  will  be  useful,  so  1 
at  all  levels  ideally  this  would  be  central  to  sharing  info  and  every  soldier  is  a 
sensor. 


Every  soldier  would  benefit  from  this  technology;  however  too  much  input  can  1 

clutter  screen  and  inhibit  effectiveness. 

With  the  networking  between  different  devices,  the  inclusion  of  symbols  from  1 

higher  echelons  might  crowd  the  screen  with  icons. 

Individual  soldier  -  movement  to  points;  fire  team  -  team  member  tracking;  squad  1 
and  platoon  -  timing/movement/tracking;  company  -  tracking;  and  above  company 
level  -  tracking. 

I  think  if  every  soldier  had  this  screen,  it  would  make  it  difficult  to  move  together.  1 
There  would  be  too  many  distractions. 

Soldiers  in  the  field  would  benefit  the  most  from  the  use  of  this  technology.  1 

Not  every  individual  needs  this.  One  leader  should  navigate  the  squad.  1 

Everyone  in  leadership  needs  to  know  what  is  going  on.  But  the  higher  ups  can  get  a  1 
brief  after  missions. 

It  would  be  good  for  squad  leaders  and  above  to  have  to  help  organize,  the  1 

individual  soldiers  should  be  paying  attention  to  what’s  around  them  not  where 
they  are;  that  is  the  leader’s  job. 

Should  go  to  the  squad  leader.  Having  soldiers  and  FT  all  with  this  technology  can  1 
break  command  and  control;  it  would  produce  confusion.  A  squad  leader  can 
adequately  gain  information  and  disperse  it  as  necessary. 

Leaders  should  be  given  the  equipment;  not  cost  effective  for  everyone  and  could  1 
lead  to  arguments  on  missions. 


7.  Would  you  use  this  technology  in  combat? 


Number  of  Responses 

No 

3 

Yes 

29 

Comments 


No.  of  Responses 


It  could  and  would  save  lives. 


2 
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Comments 


No.  of  Responses 


The  technology  seems  to  have  great  potential.  2 

If  safe  from  hackers,  this  technology  would  be  amazing  at  all  levels  of  operation.  1 

If  it  can  adequately  mark  IEDs,  caches,  HVTs,  etc.,  then  I  would  absolutely  use  it  in  1 

combat. 

If  the  platoon  leaders  transmitted  what  they  had  done  to  squad  leaders  and  they  1 

know  where  everyone’s  position  is  then  that  makes  fire  team  leader  like  a  surgeon 
when  completing  mission. 

As  a  UAV  operator,  this  would  help  us  know  exactly  where  our  friendly  units  are  1 
and  where  they  are  planning  to  go. 

Definitely  would  save  some  time  land  navigating.  Being  able  to  see  where  your  1 

soldiers  are  at.  Speed  up  the  process  of  Medevac  and  Hazmat. 

This  program  allows  the  ability  to  see  and  plan  waypoints  ahead  of  current  location,  1 
integrating  a  real  view  and  map/topo  into  one  view  as  if  on  the  ground. 

Yes,  for  planning  and  at  the  team/squad/PL  level  for  movement/tracking;  1 

PL/company  level  for  tracking;  group/formation  movement,  tracking  and  planning. 

This  would  be  a  lot  better  than  guessing  where  a  person  is  on  a  map.  This  gives  real  1 
time  location  at  all  times.  This  rocks  until  the  battery  fails. 

If  my  options  are  this  or  a  regular  map,  I’ll  easily  choose  this  technology.  Only  a  1 
fool  (no  offense)  would  take  the  paper  map. 

Yes,  would  make  routes  easier  to  follow  and  save  time  during  movement.  1 

Would  be  useful  for  finding  the  objective.  1 

I  would  feel  comfortable  using  2D-NU  to  navigate.  1 

Would  be  useful  for  squad  waypoints  to  rally  or  flank.  1 

Although  the  AR  is  in  general  less  practical  than  the  other  views,  it  would  be  useful  1 
if  you  could  toggle  it  for  reasons  such  as  marker  placement  or  route  identification. 

AR  would  make  me  nervous  if  I  had  to  use  it  in  combat.  1 

The  AR  would  be  100%  better  if  you  had  a  map  reference  as  to  your  current  location  1 
and  surroundings. 

I  would  find  it  difficult  with  adrenaline  pumping  or  explosions  going  off  to  carefully  1 
place  my  fat,  shaking  finger  perfectly  on  target  location.  I  feel  a  little  stick  might 
help  this  problem. 

Special  training  for  situational  awareness  would  need  to  occur  for  soldiers  to  2 

effectively  use  this  in  combat. 

If  there  are  route  lines,  soldiers  will  only  pay  attention  to  that.  This  means  they  are  1 
vulnerable  for  different  attacks. 

No  confusion  of  path,  what  is  going  to  be  encountered  (in  terms  of  terrain  and  1 

environment)  and  share  vital  info  at  a  moment’s  notice. 

Focus  on  digital  lines  and  emblems  would  distract  from  concentration.  1 

Not  yet.  Makes  orienting  a  bit  easier,  but  not  at  all  essential.  Simple  GPS  display  is  1 
all  I  would  use. 

It  would  help  in  getting  from  one  location  to  another  very  quickly,  but  it  could  be  a  1 
problem  if  soldiers  become  too  reliant  on  the  system  and  not  what  is  happening 
around  them. 
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Comments 


No.  of  Responses 


Yes,  but  for  short  periods  of  time.  It’s  too  slow  and  doesn’t  mark  targets  specifically  1 


unless  you  spend  a  lot  of  time  to  accurately  plot  it. 

Yes,  but  only  slightly.  It  takes  too  much  attention  away  from  immediate  1 

surroundings. 

If  it  worked  great.  1 

Needs  to  be  more  refined.  1 

Depends  on  the  level,  scope,  and  intensity.  In  a  firefight,  I  would  not  be  pulling  it  1 


out  constantly.  But  to  mark  a  pickup  site  or  a  call  for  fire,  it  can  be  helpful  if  it  can 
be  used  in  a  short  time. 

Lag  time  would  need  to  disappear  and  the  chevron’s  position  on  the  map  would  need  1 
to  more  accurately  reflect  its  location.  For  example,  there  were  several  instances 
where  my  chevron  was  located  inside  a  building,  when  I  was  most  certainly  not. 

Made  moving  down  various  alleyways  a  little  confusing,  especially  in  2D  modes. 

8.  Which  way  would  you  like  to  see  entities  rendered? 


Number  of  Responses 

Floating 

6 

Laying  on  the  ground 

8 

Lollipop  style 

15 

Other  (specify  below) 

1 

NR 

2 

Comments  No.  of  Responses 


Floating 

Would  be  easiest,  I  believe.  1 

Floating  blocks  out  a  fair  amount  of  the  screen  when  you  get  close.  1 

You  can  see  them  from  farther  away  and  more  clearly  than  if  they  are  lying  on  the  1 
ground. 

Floating  but  with  user  changeability  of  size,  shape,  style.  1 

Ground 

Unreadable.  1 

If  2D-NU  and  2D-R,  then  lying  on  the  ground  will  be  convenient  enough.  1 

Lollipop 

Works  great  and  you  can  see  and  make  pop-up  targets.  1 

The  lollipop  allowed  me  to  see  the  point  or  building  along  with  the  identifiers.  1 

Makes  it  very  clear  which  point  it’s  referring  to.  If  there  are  two  thick  points  1 

together,  it  can  be  hard  to  distinguish  which  is  which. 

The  lollipop  style  lets  you  know  exactly  where  the  target  is,  so  no  guessing  is  1 

required. 
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Comments 


No.  of  Responses 


Easier  to  match  with  buildings  and  items.  1 

Lying  on  the  ground  worked  best  in  2D-R  and  2D-NU  scenarios.  1 

Best  in  AR  style.  3 

It  would  be  better  for  AR  mode.  There  is  little  reason  to  change  entity  rendering  for  1 
the  other  maps. 

AR  or  3PP,  which  will  allow  you  to  see  the  exact  location  of  the  object.  1 

For  the  2D  maps,  this  doesn’t  make  sense  (and  since  I  preferred  the  2D  maps,  lying  1 

on  the  ground  is  best). 

As  long  as  “stick”  shows  exactly  what  is  being  labeled  so  it  is  easy  to  understand  1 
which  object  is  being  marked. 

Lollipop/floating  might  work  for  symbols/targets  located  much  farther  away  from  1 
present  location. 

General  comments 

Not  important;  any  would  work.  1 

The  3PP  needs  to  be  improved.  It  reminds  me  of  Nintendo  graphics  where  at  times  1 
you  can’t  see  a  point  due  to  lagging  until  it  is  too  late. 

The  2D-R  seems  to  be  the  most  proactive  feature  at  this  time.  1 

The  pie  map  needs  to  have  the  option  of  N  stationary  and  rotating.  I  would  also  like  1 
the  pie  to  have  the  ability  to  zoom  in  or  out  depending  on  needs  of  terrain  and 
urban  style. 

Whatever  method  is  chosen,  entities  should  be  dramatically  reduced  in  size  as  the  1 
person  navigating  approaches  them. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


2-D 

two-dimensional 

2D-NU 

2D  north  up 

2D-R 

2D  rotating 

3-D 

three-dimensional 

3PP 

third-person  perspective 

ANOVA 

analysis  of  variance 

AR 

augmented  reality 

CCP 

casualty  collection  point 

CFF 

call  for  fire 

EUD 

end  user  device 

GPS 

global  positioning  system 

IED 

improvised  explosive  device 

NASA 

National  Aeronautics  and  Space  Administration 

NIE 

Network  Integration  Evaluation 

NSRDEC 

Natick  Soldier  Research,  Development  and  Engineering  Center 

OC 

observer  controller 

OCS 

Officer  Candidate  School 

RT 

response  time 

TLX 

Task  Load  Index 
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NO.  OF 

COPIES  ORGANIZATION 

1  DEFENSE  TECHNICAL 

(PDF  INFORMATION  CTR 
only)  DTIC  OCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BELVOIR  VA  22060-6218 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMAL  HRA 

2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
RDRL  CIO  LL 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
RDRL  CIO  LT 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 
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NO.  OF 

COPIES  ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  C  A  DAVISON 

320  MANSCEN  LOOP  STE  1 15 
FORT  LEONARD  WOOD  MO  65473 

2  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  DI 

T  DAVIS 

J  HANSBERGER 

BLDG  5400  RM  C242 

REDSTONE  ARSENAL  AL  35898-7290 

1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HRS  EA  DR  V  J  RICE 
BLDG  4011  RM  217 
2377  GREELEY  RD  STE  R 
FORT  SAM  HOUSTON  TX  78234-7731 

1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HRM  DG 
I  RUBINSTEIN 
BLDG  333 

PICATINNY  ARSENAL  NJ  07806-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ARMC  FIELD  ELEMENT 
RDRL  HRM  CH  C  BURNS 
THIRD  AVE  BLDG  1467B  RM  336 
FORT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 

AWC  FIELD  ELEMENT 
RDRL  HRM  DJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FORT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HRM  CK  J  REINHART 
10125  KINGMAN  RD  BLDG  317 
FORT  BELVOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HRM  AY  M  BARNES 
2520  HEALY  AVE 
STE  1172  BLDG  51005 
FORT  HUACHUCA  AZ  85613-7069 

1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HR  MP  D  UNGV ARSKY 
POPE  HALL  BLDG  470 
BCBL  806  HARRISON  DR 
FORT  LEAVENWORTH  KS  66027-2302 


1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AT  J  CHEN 
12350  RESEARCH  PKWY 
ORLANDO  FL  32826-3276 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AT  C  KORTENHAUS 
12350  RESEARCH  PKWY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AS  C  MANASCO 
SIGNAL  TOWERS 
BLDG  29808A  RM  303 
FORT  GORDON  GA  30905-5233 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  CU 
6501  E  1 1  MILE  RD  MS  284 
BLDG200A  1ST  FL  RMA1117 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
FIRES  CTR  OF  EXCELLENCE 
FIELD  ELEMENT 
RDRL  HRM  AF  C  HERNANDEZ 
3040  NW  AUSTIN  RD  RM  221 
FORT  SILL  OK  73503-9043 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AV  S  MIDDLEBROOKS 
91012  STATION  AVE 
FORT  HOOD  TX  76544-5073 

1 0  ARMY  RSCH  LABORATORY  -  HRED 

HUMAN  RSRCH  AND  ENGRNG 
DIRCTRT  MCOE  FIELD  ELEMENT 
RDRL  HRM  DW  E  REDDEN 
7533  HOLTZ  ST. 

BLDG  70  RM  1056 

FORT  BENNING  GA  31905-5400 
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NO.  OF 

COPIES  ORGANIZATION 

1  ARMY  G1 

(CD  DAPE  MR  B  KNAPP 

only)  300  ARMY  PENTAGON  RM  2C489 

WASHINGTON  DC  20310-0300 

1  NATICK  SOLDIER  CENTER-NSRDEC 

RDNS  TSI  D  DARKOW 
F-118,  BUILDING  FIVE 
15  KANSAS  STREET 
NATICK,  MA  01760-2642 

1  NATICK  SOLDIER  CENTER-NSRDEC 

RDNS  TSI  D.  MAGNIFICO 
F-118,  BUILDING  FIVE 
15  KANSAS  STREET 
NATICK,  MA  01760-2642 

1  NATICK  SOLDIER  CENTER-NSRDEC 

RDNS  TSI  N  SOTO 
F-118,  BUILDING  FIVE 
15  KANSAS  STREET 
NATICK,  MA  01760-2642 

ABERDEEN  PROVING  GROUND 


4  DIR  USARL 
RDRL  HR 
L  ALLENDER 
T  LETOWSKI 
RDRL  HRM 

P  S  A V AGE-KNEPS  HIELD 
RDRL  HRS  D 
B  AMREIN 


63 


Intentionally  Left  Blank. 


64 


